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INTRODUCTION 

In  a  separate  policy  paper  for  the  Ontario  Economic  Council  entitled 
The  Social  Service  System  in  Ontario  (McCready-Rnhn ,  1983),  the  authors 

have  addressed  possible  sources  of  motivation  within  the  Ontario  social 
service  system  with  a  view  to  the  question  of  aoeountabil  ity .  The 
authors  concluded  that  to  tie  together  economic  costs  with  treatment 
outcomes,  coupled  with  greater  community  knowledge  of  problem  rates  and 
problem  severity  would  permit  the  government  and  the  United  Way  to  allo¬ 
cate  scarce  resources  more  optimally. 

In  the  policy  paper,  the  Ontario  social  service  system  is  found  to 
have  been,  in  19S0-81,  a  $951,205,716  industry.  In  that  fiscal  year,  it 
is  estimated  tha t  87%  of  those  funds  came  from  federal,  provincial,  and 
municipal  taxes,  10.3%  from  fees  and  endowment,  and  2.7%  from  the  United 
Wa  y . 

With  respect  to  administrative  auspice  for  the  delivery  of  social 
services,  49%  of  the  services  were  administered  by  voluntary,  non-profit 
agencies,  43%  by  government  departments,  and  8%  by  prof it-or iented  firms, 
chiefly  commercial  day  care  centres. 

Also  discussed  in  the  policy  paper  are  some  theoretical  issues  which 
have  to  do  with  the  nature  of  the  market  for  social  service  goods,  maxi¬ 
mization  options  for  social  service  organize tions ,  and  certain  methodo¬ 
logical  trends  around  applied  research  for  agency  information  system 
devel opment . 

The  paper  concludes  that  unit  cost  data  alone  are  misleading  both 
for  internal  agency  planning  and  for  external  funding  purposes.  True 
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efficiency,  of  course,  has  to  do  with  optimizing  the  combination  of  best 
outcomes  for  least  costs.  But  the  subject  of  cost-outcome  and  cost- 
effectiveness  analysis  lias  been  too  long  debated  and  too  little  demon¬ 
strated.  In  conjunction  with  the  policy  study,  it  was  decided,  there¬ 
fore,  to  undertake  some  feasibility  demonstrations  in  Ontario  social 
service  programs. 

Findings  from  the  Six  Demonstrations 

The  six  field  demonstrations  do  confirm  the  feasibility  of  moving 
from  service  unit  cost  (cost  per  day,  cost  per  treatment  hour,  cost  per 
home  visit)  to  cost  per  closed  case  and  on  to  a  cost-outcome  analysis. 

Each  setting  was  unique  with  respect  to  the  information  elements 
already  available  for  cost-outcome  analysis  work.  Each  setting  differed 
as  to  where  it  was  on  the  evolutionary  scale  of  information  system  devel¬ 
opment.  In  all  six  settings,  thought  had  been  given  and  was  currently 
being  given  to  the  subject  of  information  system  development,  often  in 
response  to  initiatives  by  the  Ontario  Ministry  of  Community  and  Social 
Services  but  also  in  response  to  a  professional  interest  and  awareness  of 
the  importance  of  the  subject  among  management,  supervisory,  and  line 
personnel . 

Given  the  current  emphasis  on  information  system  development  wi thin 
the  Ministry,  the  next  few  years  my  see  major  advances  in  this  regard. 
The  crucial  issues  will  be  methodological,  not  technical;  that  is,  what 
information  should  be  captured  is  the  central  ijuestion.  If  the  informa¬ 
tion  requirements  for  cost-outcome  analysis  are  included  for  attention, 
it  should  cost  no  more  for  agencies  to  develop  an  information  system 
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which  includes  a  cost-outcome  analysis  capacity  -  than  to  develop  one 

without  it. 

At  a  methodological  level/  the  rel iabil i ty  of  a  treatment  outcome 
measurement  scale  improves  as  one  moves  from  a  subjective  problem 
oriented  scale  format  to  objective  goal  attainment  scaling,  single  sub¬ 

ject  charting,  or  the  use  of  a  stnndardired  developmental  inventory 
screen.  But  all  of  the  scaled  measurement  options  are  better  than  no 
measurement  at  all.  The  val  idity  of  the  treatment  outcome  data  also 

varies  among  the  several  scale  options  illustrated  in  our  six  settings, 
but  the  validity  issue  is  getting  lots  of  attention  in  the  literature, 
and  progress  is  being  made.  * 

Even  without  objective  outcome  scaling,  there  appears  to  be  poten¬ 
tial  utility  in  having  cost  and  outcome  data  for  single  cases.  This  kind 
of  information  is  never  available  in  a  one-shot  inferential  level  re¬ 

search  study  using  experimental  and  control  or  comparison  groups.  We 
have  noted  the  possible  utility  of  special  file  studies  on  cases  present¬ 
ing  a  particular  type  of  problem,  especially  whore  low  success  in  rela¬ 
tion  to  cost  is  revealed  and  some  innovation  in  approach  to  treatment 
seems  to  be  needed. 


*  The  more  objective  outcome  measures  lend  themselves  best  to  com¬ 
parisons  of  cost-outcome  between  alternative  programs.  The  subjectivity 
of  many  scales  introduces  an  element  of  non-comparability.  The  highly- 
dcsirable  objective  is  to  develop  measures  of  cl ient  functioning  at  in¬ 
take  and  at  closing  which  can  be  subject  to  operational  audit.  For  an 
authoritative  review  of  the  fast  emerging  technology  around  treatment 
outcome  measurement  and  the  specification  of  1 ntervent ions ,  see  Martin 
Bloom  and  Joel  Fischer,  Evaluating  Practice  (Guidelines  for  the  Account¬ 
able  Professional),  (Englewood  Cliffs,  Frentioe-llal  1  ,  1'kC)  . 
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There  is  also  the  possibility  that  replication  among  im  tolled  clients 
over  several  years  can  offer  useful,  though  not  fully  comparable,  find¬ 
ings  as  a  substitute  for  random  group  designs  in  program  evaluation.  The 
data  on  cost  and  outcome  for  single  cases  as  well  as  for  aggregated  cases 
in  our  demonstrations  further  confirms  the  feasibility  of  replication 
work,  though  at  a  lesser  level  of  rigor  than  for  inferential,  group 
design  studies. 

In  addition  to  the  need  for  progress  in  the  use  of  treatment  outcome 
scales  by  front  line  workers,  there  are  two  additional  aspects  of  in¬ 
formation  system  development  which  need  special  attention:  (1)  worker 
time  and  activity  data,  (2)  the  definition  and  standardiza tion  of  treat¬ 
ment  modalities. 

Most  line  workers  laave  had  highly  exasperating  experience  with  daily 
log  procedures  and  the  necessity  of  summarizing  such  data  at  the  end  of 
each  week  or  month.  They  are,  therefore,  very  suspicious  of  the  whole 
notion  of  worker  time  use  reporting.  Careful  design  work  for  a  computer¬ 
ized  format  with  coded  information  categories  in  many  agonies  remains  to 

be  done.  The  format  should  require  no  more  than  five  minutes  a  day  from 

the  line  worker  and  should,  of  course,  be  tabulated  entirely  by  machine 

or  by  an  information  clerk  so  that  none  of  the  compilation  has  to  be  done 

by  the  line  worker. 

The  question  of  how  best  to  define  and  standardize  various  treatment 
modalities  is  the  newest  and  least  resolved  question  in  social  work  and 
mental  health  practice.  It  is  perhaps  the  central  issue  currently  in  all 
quality  assurance  procedures  and  formats,  outside  of  medicine.  Treatment 


modalities  such  as  supportive  counselling,  psychosocial  treatment,  family 
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unit  treatment,  reality  therapy,  transactional  analysis,  or  behaviour 
modification  could  be  coded  and  captured  for  single  cases  in  a  computer¬ 
ized  information  system.  This  would  permit  some  aggregate  case  analysis 
of  cost  and  outcome  for  certain  kinds  of  problems  using  defined  and  dif¬ 
ferential  approaches  to  treatment. 

The  problem  of  how  to  define  and  standardize  the  type  of  treatment 


given  has 

long  been 

a  vital  issue 

in  inferential 

group  design  program 

eval ua  tion 

studies . 

It  is  just  as 

vital  in  single 

subject  charting  and 

the  aggregation  of  single  case  data  for  an  ongoing  agency  data  base. 

Judging  from  the  letters  of  comment  from  the  six  administrators  in¬ 
volved  (see  Appendices  A  to  F)  ,  and  from  our  own  observation,  cost-out¬ 
come  analysis  is  feasible.  If  the  Ministry  of  Community  and  Social 
Services  should  decide  to  use  cost-outcome  data  as  a  measure  of  effect¬ 
iveness  and  efficiency  in  shaping  payments  to  operating  agencies,  there 
is  no  reason  to  believe  it  could  not  be  done.  It  would  be  important  to 
ensure  that  sufficient  time  and  consultation  were  available  to  permit  the 
operating  agencies  to  develop  the  information  system  components  needed 
for  cost-outcome  work  over  a  reasonable  period  of  time  and  at  a  sensible 
pace.  Much  basic  information  system  development  is  already  underway  with 
Ministry  help  and  encouragement.  By  and  large,  the  only  crucial  informa- 
tion  items  which  would  liave  to  be  added  to  the  present  contemplated  code 
lists  would  be  worker  time  by  case  number  (as  well  as  program),  some  cod¬ 
ing  of  treatment  modalities,  and  better  scaled  treatment  outcome  record¬ 
ing  on  client  file  face  sheets.  The  hurdle  is  not  cost,  but  conceptual . 

There  is  and  Ins  been  much  discussion  but  little  evidence  of  the' 
feasibility  of  undertaking  cost-effectiveness  analysis  in  social  service 
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agencies.  As  a  consequence,  in  carrying  out  these  Ontario  Economic 
Council  commissioned  studies  of  non-profit  and  governmental  social 
services,  the  researchers  invited  the  administrators  of  six  diversified 
agencies  to  collaborate  in  field  demonstrations. 

General  agreement  to  participate  was  reached  with  the  six  adminis¬ 
trators  in  July  1981  with  the  understanding  that  the  installation  of 
whatever  worker  time  sheets  and  client  file  treatment  outcome  sheets 
needed  would  be  worked  through  in  August  or  September,  followed  by  data 
gathering  on  a  few  cases  over  a  limited  period  of  time  between  September 
1981  and  April  1982. 

Early  on,  it  was  discovered  that  in  some  instances  the  cost-outcome 
or  cost-effectiveness  study  could  be  done  with  existing  data  on  a  retro¬ 
spective  basis;  in  some,  a  prospective  study  was  indicated.  In  a  few  of 
the  situations,  the  current  treatment  outcome  scale  instruments  already 
in  place  could  be  used;  in  other  situations,  a  particular  option  for 
outcome  measurement  was  selected  and  installed  col  labora  tivel  y .  The 
picture  which  emerged  is  shown  in  Table  1.  All  six  cost-outcome  studies 
were  agency  data  base  studies;  none  were  in-depth,  inferential  level, 
evaluative  research  studies. 

Insert  Table  1  here 

The  stated  purpose  of  each  of  the  six  field  studies  wus  to  demon¬ 
strate  the  fea  sibil  i  ty  of  the  data  gathering  procedures  required,  not  to 
generate  firm  cost-outcome  or  cost-effectiveness  findings  for  formal  use 
in  internal  program  development  or  external  funding  submissions  at  this 
point.  It  would  be  cavalier  indeed  to  presume  to  see  six  agencies  in- 
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TABLE  INT.-1 

CHARACTERISTICS  OF  SIX  COST  OUTCOME  TRIALS 


Program 

Type 

Admi nis- 

tra  tive 
Auspice 

Retrospective 

or 

Prospective 

Cost -out come 

or  Cost- 
ef f ectivcness 

Outcome  Scale 
Option  Used 

1 .  Chil dren ' s 

Mental  Health 

Centre 

Nonprof  it 

Retrospective 

C-E 

Goal  attain¬ 
ment  seal ing 
(obj  ect ive ) 

2.  Vocational 

Rehabil  i- 

ta  tion 

Publ ic 

Retrospective 

C-0 

Goal  attain¬ 
ment  scaling 
(dichotomized ) 

3.  Preschool 

Publ ic 

Prospective 

C-0 

Dev el opme nta 1 
inventory 

4.  Group  Home 

Non¬ 

profit 

Retrospective 

C-0 

Subj  ective, 
goal  oriented 
seal  e 

5.  Children's 

Aid  Society 

Non¬ 

profit 

Prospective 

C-0 

Subj  ective , 
probl em- 
oriented  scale 

6.  Charitable 

Home  for  the 
Aged 

Nonprofit 

Retrospective 

C-0 

Subj  ective , 
probl em- 
oriented  scale 
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stall  and  operationalize  a  permnont,  on-going  cost-outcomes  analysis 
capacity  with  less  than  two  months  of  consul ta tion  and  development  time. 

The  six  field  demonstrations  are  more  in  the  nature  of  preliminary 
pilot  studies  which  can  further  the  orientation  of  administrators  and 
staff  as  to  what  would  be  involved  in  moving  toward  a  cost-effectiveness 
analysis  capacity. 

The  findings  from  each  of  the  six  locations  are  now  presented. 
Comments  on  the  studies  and  the  approach  follow  the  erase  studies. 

LUTHERWOOD,  INC. 

(Childrens'  Mental  Health  Centre) 

Lutherwood  is  a  non-profit  children's  mental  health  centre  organized 
in  1970  and  sponsored  by  the  Ontario  District  of  the  Lutheran  Church 
Missouri  Synod.  It  is  located  a  few  miles  north  of  Waterloo,  Ontario  in 
a  wooded  and  open  country  setting.  It  is  operated  by  a  Board  of  Direct¬ 
ors  of  17  persons  who  are  elected  by  the  Ontario  District  of  the  Synod; 
60%  are  to  be  church  members  within  the  Missouri  Synod.  In  addition  to 
the  Board  of  Directors,  the  agency  Iras  a  Professional  Advisory  Committee 
of  10  persons  including  specialists  from  social  work,  clinical  psych¬ 
ology,  psychiatry,  theology,  kineseology,  child  care  and  education. 

« 

Lutherwood  in  1982  operated  four  discrete  programs:  (1)  a  residen¬ 
tial  cottage  program  (capacity-14);  (2)  a  day  school  ("day  growth")  pro¬ 
gram  ( cape  city-20 )  ;  (3)  an  out-patient  clinic  (initiated  in  September 

1981);  and,  (4)  a  demonstration  program  in  neighborhood  development  and 
primary  or  secondary  prevention  at  a  decentral ized  location  in  Kitchener 
with  a  neighborhood  population  of  4000. 
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Funding  for  Lutherwood  programs  is  on  the  hi  sis  of  100%  from  the 
Ontario  Ministry  of  Community  and  Social  Services  for  operating  expendi¬ 
tures.  For  capital  expenditures,  one-third  of  the  funds  are  privately 
raised  and  two-thirds  come  from  the  Ministry,  although  in  recent  years 
the  agency  has  been  providing  more  than  one-third  of  new  capital  cost. 

In  1974,  Lutherwood  installed  the  Kiresuk  -  Sherman  Coal  Attainment 
Scaling  (GAS)  technology  for  treatment  planning  and  for  case  outcome  and 
program  evaluation.  One  full  time  position  is  designated  to  coordinate 
and  maintain  the  system. 

The  GAS  system  yields  a  score  which  Gin  range  from  about  20  to  80. 
If  a  client  is  functioning  at  the  "expected  level  of  success"  at  dis¬ 
charge  or  at  three  month  follow-up,  the  score  is  50.  Each  evaluation 
guide,  developed  at  intake  and  used  to  rate  outcome  at  discharge  and 
three  month  follow-up,  usually  focusses  on  from  two  to  five  major  problem 
and  goal  areas.  A  client  at  discharge  might  be  functioning  above  50  in 
some  goal  areas,  and  below  50  in  others,  but  the  overall  "average"  goal 
achievement  score  could  still  be  50. 

The  way  Lutherwood  measures  the  scores  at  intake,  using  the  current 
position  of  the  client  as  the  worst  possible  outcome,  ensures  that  the 

admitting  score  will  be  in  the  19-29  range.  An  examination  of  the  sever- 

« 

ity  of  presenting  problems  would  require  an  objective  score  on  admission 
according  to  some  norm.  Although  many  questions  oannot  be  answered  be¬ 
cause  of  Lutherwood '  s  method  of  scoring,  it  does  not  preclude  a  cost- 
outcome  analysis. 

Since,  at  Lutherwood,  GAS  score  data  are  routinely  obtained  and  re¬ 
corded  for  each  young  person  in  treatment,  it  was  possible  to  use  a 
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retrospective  approach  so  far  as  treatment  outcome  data  were  concerned. 
It  was  agreed  to  focus  on  just  two  programs,  the  cottage  program  and  day 
school  program  and  cost  data  were  available  on  the  basis  of  cost-per-day . 

There  is  a  weakness  in  global  per-diem  cost  da  ta  in  that  direct 
treatment  cost  cannot  be  examined  as  separate  from  administrative  cost; 
nor  can  the  components  of  direct  treatment  cost  be  examined  differential¬ 
ly.  Lutherwood,  is,  however,  refining  its  program  and  cost  information 
gathering  system  to  get  at  such  internal  performance  and  cost  data.  Per- 
diem  cost  data,  however,  do  include  all  relevant  treatment  and  admin¬ 
istrative  costs.  Social  costs  and  opportunity  costs  are  not  included. 
For  cost-outcome  analysis  of  programs  involving  young  people  below  regu¬ 
lar  working  ages  there  may  be  justification  in  not  including  the  oppor¬ 
tunity  costs  but  social  costs  properly  ought  to  be  included. 

Table  1-1  presents  raw  data  for  GAS  scores  at  intake,  discharge,  and 
three  month  follow-up  along  with  length  of  stay,  for  10  cottage  program 
cases  and  10  day  school  cases.  The  costs  per  unit  of  change  have  also 
been  calculated.  It  is  evident  that  the  costs  of  the  court-appointed 
cases  are  higher.  For  instance,  the  average  cost  per  unit  of  change  of 
the  voluntary  admissions  is  $1,100.22  while  the  average  cost  per  unit  of 

change  for  the  court  admissions  is  $3,470.S4  in  the  cottage  program.  In 

« 

the  day  school,  the  same  costs  are  $420.40  and  $1,544.43  respectively. 

Insert  Table  1-1  here 

Cases  C-9,  C- 1 0 ,  D-9,  and  D-10  represent  admissions  under  court 

authority,  the  remainder  represent  voluntary  admissions  involving  a  pro¬ 
cess  of  negotiation  and  joint  problem  identif ica tion  in  which  the  young 


TABLE  1-1 


RAW  DATA  ON  GAS  SCORES  AND  LENGTH  OF  STAY 
Ten  Cottage  and  Ten  Day  School  Cases  (1979-81)-  Lutherwood 
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( 

:l  ient 

Intake 

GAS 

at 

Discha  rge 

Scores 

at 

Follow-up 

Cha  ngt 
a  t 

Discha  rgt 

;  Scores 

a  t 

a  Follow-up 

Length  of 
Stay 
in  da y s 

Cost 

COTTAGE 

( 

C-1 

19.85 

50.00 

50.00 

30.  15 

30.  15 

1  19 

13,715.94 

C-2 

19.85 

74 . 12 

50.00 

54.27 

30  .  15 

420 

47,166.00 

C-3 

19.85 

53.62 

51.81 

33.77 

31.96 

382 

42,673.96 

u 

C-4 

19.02 

57.75 

65.49 

38.73 

46.47 

316 

34,992.49 

C-5 

19.85 

59.05 

68.09 

39.20 

48.24 

260 

28,662.24 

f 

C-6 

19.35 

74.12 

74 . 12 

54.27 

54.27 

359 

39,566.82 

! 

C-7 

19.85 

59.05 

59.05 

39.20 

39.20 

339 

37,385.94 

C-8 

19.85 

37.94 

50.00 

18.09 

30 .15 

486 

52,399.96 

(1  C_9 

19.92 

34.51 

47.38 

14 . 59 

27.46 

754 

80,245.  15 

1  C-10 

19.82 

46.98 

34.92 

27.  16 

15.  10 

364 

37,764 . 03 

_  Averaqe 

j 

19.77 

54.71 

55.09 

34.94 

35.32 

381 

41,457.26 

u 

DAY  SCHOOL 

I]  D-1 

19.90 

65.73 

56. 16 

45.83 

36.26 

367 

17,066.48 

D-2 

19.85 

68.09 

62.02 

48.24 

42. 17 

400 

18, 149.22 

n 

D-3 

19.85 

37.94 

59.05 

18.09 

39.20 

259 

11,251.51 

i 

D-4 

19.92 

50.00 

57.85 

30.00 

37.93 

170 

7,539.44 

D-5 

19.85 

68.09 

71.11 

48.24 

51.26 

377 

16,833.27 

| 

D-6 

19.80 

50.00 

60.78 

30.20 

40.98 

404 

17,537.90 

D-7 

19.92 

70.93 

74.85 

51.01 

54.93 

446 

19,426.04 

II 

D-8 

19.92 

54.58 

53.27 

34.66 

33.35 

349 

14,931.61 

D-9 

21  .  16 

48.97 

50.00 

27.81 

28.84 

326 

15,  104.31 

o 

i 

o 

19.85 

28.89 

46.99 

9 . 04 

27  .  14 

502 

22,987.39 

Average 

20.00 

54.32 

— 

59.21 

34.31 

— 

39.21 

360 

16,090.62 

Cost/Unit 
Change  in 
Score 


'154.92 
869. 10 
1,263.66 
903.50 
731.18 
729.07 
953.72 
2,896.63 
5,592.00 
1 , 349.68 

1,574.35 


372.39 

376.23 

621.97 

251.28 

348.95 

580.73 

390.83 

430.80 

546.00 

2,542.85 

645.20 
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person  and  parents  were  involved.  The  20  cases  examined  were  the  most 
recently  discharged  cases. 

Ministry  approved  per-diem  rates  for  the  period  ranged  from  $101.42 
to  $126.95  in  the  cottage  programs  and  from  $45.24  to  $58.31  in  the  day 
school  program.  The  cottage  program  runs  seven  days  a  week;  the  day 
school  program,  five  days  a  week. 

For  the  analysis  in  Table  1-2  belowT,  four  alternative  definitions  of 
success  are  exercised: 

(1)  a  change  score  of  25  points  or  more  at  discharge; 

(2)  a  change  score  of  25  points  or  more  at  three  month  follow-up; 

(3)  a  GAS  score  of  50  or  more  at  discharge;  and, 

(4)  a  GAS  score  of  50  or  more  at  follow-up. 

To  properly  compare  the  Cottage  Program  with  the  nay  School  Program 
the  young  clients  would  have  to  be  randomly  allocated  to  each  program,  or 
a  matched  pairs  procedure  might  be  used.  In  a  retrospecitve  study, 
neither  is  possible.  The  presenting  scores  are  similar  because  of 
Lutherwood  using  the  current  functioning  of  the  client  as  the  worst  pos¬ 
sible  outcome  and  there  are  other  characteristics  such  as  the  home  situa¬ 
tion  which  would  be  necessary  in  comparing  the  two  programs. 

« 

Insert  Table  1-2  lie  re 

Table  1-2  presents  an  analysis  for  cottage  program  cases  for  the 
eight  cases  involving  voluntary  admissions  as  compared  to  ten  cases  when 
the  two  court  ordered  oases  are  included.  The  agency  was  interested  in 
such  an  analysis  because  the  staff  had  felt  for  some  years  that  the  level 
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of  success  with  court-ordered  cases  was  well  below  that  for  cases  involv¬ 
ing  a  voluntary  admission.  Feedback  of  this  kind  to  individual  front 
line  workers,  supervisors,  and  management  would  appear  to  have  utility  in 
searching  out  both  the  client  characteristics  and  program  elements  which 
may  be  "associated",  on  an  eye-ball  and  non-sta ti stical  basis,  with  im¬ 
proved  levels  of  success.  When  and  if  some  good  operational  hunches  are 
sorted  out,  they  ran  be  tried  out,  again  on  an  aggregation  of  single  case 
data  basis. 

An  alternative  to  the  dichotomized  outcome  scale  is  to  use  the 
Soronsen-Grove  Cost-Effectiveness  Matrix.*  Tn  the  Sorensen-Grove  matrix, 
cost  is  isolated  by  several  levels  of  treatment  outcome.  Rather  than 
dichotomizing  outcome  (successful-unsuccessful),  Sorensen  and  Grove  would 
identify  cost  for  each  case  for  each  of  several  degrees  of  change  or  no¬ 
change  in  level  of  client  functioning.  For  the  Lutherwood  Day  School 
outcome  data,  the  Sorensen-Grove  Cost-Effectiveness  Matrix  might  look  as 
in  Table  1-3.  The  treatment  time  is  different  for  most  of  the  10  cases, 
ranging  from  12  to  24  months.  The  Matrix  in  Table  1-3  reports  on  all  10 
cases  including  the  two  court-ordered  admissions. 

Insert  Table  1-3  here 

« 

In  this  demonstration,  we  have  used  two  possible  approaches  to  the 
depiction  of  cost-outcome  analysis  information  have  been  identified:  (1) 
an  average  cost  per  successful  closed  case  for  all  cases  classified  as 

*  See  James  E.  Sorensen  and  Hugh  D.  Grove,  "Using  Cost-Outcome  and 
Cost-Effectiveness  Analyses  for  Improved  Trogram  Management  and  Accounta¬ 
bility"  in  Evaluation  of  Human  Service  Trcgrams,  edited  by  C.  Clifford 
Attkisson,  William  A.  Hargreaves  and  Mardi  J.  Horowitz  (New  York: 
demic  Press,  1978). 


Aca  - 


GAS  Scores-Intake 


TABLE  1-3 


COST-OUTCOME  MATRIX  FOR  LUTHEKUiOQD  DAY  GROWTH  PROGRAM 


N  =  10  cases 

GAS  Scor es-Di scha rge 


19-29  30-39  40-49  50-59  60-69  70-79 


19-29 

- —j 

O 

#3 

#  9 

#4 

#6 

#8 

#1 

#2 

#5 

#7 

30-39 

Average 

$22,987 

$11,252 

$15, 184 

$13,336 

$17,350 

$19,426 

Cost 


Marginal  -$11,735  +$3,932  -$1,846  +$4,012  +$2,076 

Cost 
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successful  or  unsuccessful  (Table  1-2),  and  (2)  the  Sorensen-Grove  matrix 
which  reports  costs  j^er  closed  case  for  each  cise,  the  successful  and  the 
less  successful  or  unsuccessful  oases,  classified  on  a  continuum  of  sev¬ 
eral  levels  of  success  and  failure  including  a  display  of  before  and 
after  level  of  functioning  for  each  case  (Table  1-3). 

It  would  seem  that  the  Sorensen-Grove  matrix  is  a  clearly  superior 
format.  Table  1-3  formats  such  a  cost-outcome  analysis  for  the  Day 
School  (Day  Growth)  program  only  but  a  similar  portrayal  for  the  Cottage 
program  could,  of  course,  be  prepared. 

However,  even  a  dichotomized  outcome  scale  depiction  comparing  the 
Cottage  and  Day  Growth  programs  (Table  1-4)  provides  the  kind  of  informa¬ 
tion  presumably  needed  for  some  management  and  board  decisions  around  the 
revision  or  cont inua tion  of  one  or  both  programs.  The  crucial  question, 
not  addressed  in  this  demonstration,  would  have  to  do  with  the  similari¬ 
ties  and  differences  around  problem  type  and  problem  severity  for  the 
young  peoxale  currently  admitted  to  these  two  programs.  The  dollar  con¬ 
trasts  in  cost  x>er  successful  closed  case  are  so  great  as  to  well  moti¬ 
vate  an  exploration  of  that  question. 

Insert  Table  1-4  here 

« 

At  Lutherwood,  since  its  treatment  outcome  scaling  procedures  are 
already  so  well  developed,  there  would  seem  to  be  little  or  no  additional 
cost  involved  in  building  cost-outcome  analysis  r exports  into  the  on-going 
information  system  report  formats.  Thei'e  are  limitations  to  using  global 
cost  x>er  day  as  the  service  unit  rather  than  sx^ecific  time  spent  on  each 
client.  Some  new  codes  capturing  worker  time  r.x'ent  in  each  of  the  two 


COST-EFFECTIVENESS:  LUTHERWOOD 
Average  Cost  per  Successful  Closed  Case  for  Two  Programs 
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Success:  GAS  score  of  50  or  more  at  follow-up 
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programs  more  accurately  might  need  to  be  added  to  present  worker  time 
sheets . 

It  would  appear  that  Lutherwood  could  move  to  a  comparative  cost- 
effectiveness  analysis  capacity  for  its  Cottage  and  Day  Growth  programs 
with  relatively  little  difficulty  by  adding  a  standardized  test  of  prob¬ 
lem  severity  at  intake  and  by  individualizing  cost  per  case. 

VOCATIONAL  REHABILITATION  PROGRAM 
(Ontario  Ministry  of  Community  and  Social  Services) 

The  Waterloo  Area  Office  of  the  Ministry  serves  four  county  terri¬ 
tories:  Waterloo,  Wellington,  Bruce  and  Grey.  While  for  purposes  of 

province-wide  planning  and  policy  development,  the  former  Children's 
Division  and  Adult  Division  were  integrated  in  1981  at  the  Queen's  Park 
level  of  Ministry  structure,  at  the  Area  Office  level  they  are  only 
gradually  being  integrated. 

Under  the  Adult  Services  Division  at  the  Area  Office  level-,  the 
Ministry  carries  direct  administrative  responsibility  for  any  Schedule  I 
Mental  Retardation  Facilities  located  in  Area  Office  territory,  the  in¬ 
come  maintenance  program  under  the  Family  Benefits  Act  and  for  the 

Vocational  Rehabilitation  Program  under  the  Vocational  Rehabilitation 

« 

Services  Act.  Any  Schedule  I  mental  retardation  facilities  located  in 
the  Area  Office  territory  are  administered  directly  by  the  Regional 
Office  Director.  For  all  other  adult  social  services,  non-profit, 
profit-oriented  or  municipal  agencies  deliver  the  service  usually  under 
some  kind  of  a  contract  or  legislated  funding  subsidy  from  the  Min¬ 
istry's  Area  Office  Adult  Services  Division  Manager. 
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However,  even  for  the  Ministry  administered  Vocational  Rchabil  ita- 
tion  Program,  only  the  Rehabilitation  Counsellors  are  employed  directly 
by  the  Ministry;  much  of  the  assessment  service  and  all  of  the  restora¬ 
tion  (medical,  dental,  optometric,  hearing,  prosthetic)  and  training 
services  are  provided  by  the  Ministry  under  contracts  with  nonprofit  and 
profit-oriented  workshop  agencies,  restoration  supply  providers,  com¬ 
munity  colleges,  universities,  and  occasionally,  employers.  A  special¬ 
ized  job  placement  service  is  also  provided  by  a  Ministry  employed 
Counsellor. 

The  Rehabilitation  Counsellors  are  primary  workers  or  erase  managers 
responsible  for  intake,  the  coordination  of  assessment  work,  the  pur¬ 
chase  of  restorative  and  training  services,  job  placement  and  the  moni¬ 
toring  of  all  work  around  each  case.  Completed  cases  are  followed  for 
three  months  and  then  closed.  All  closed  erases  are  followed  up  at 
twelve  months  past  closure.  The  Ministry's  job  description  for  the 
Rehabilitation  Counsellor  includes  the  following  statement  of  purpose 
for  the  position: 

To  rehabilitate  persons  whose  physical,  mental  and  emo¬ 
tional  disabilities  hinder  their  gainful  employment  and 
contribution  to  society  through  (a)  a  program  of  ass¬ 
essing  their  vocational  potential,  recommending  and 
engaging  training  and  other  vocational  services  and 
placements,  (b)  the  development  of  new  and  existing 
vocational  resources,  and  (c)  pa rticipa tion  in  program 
and  policy  development. 

For  the  purposes  of  our  cost-outcome  feasibility  denonstra  tion, 
arrangements  were  nude  through  the  Area  Manager  for  Adult  Services  for 
access  to  data  on  77  vocational  rehabilitation  cases  closed  during  the 
period  September  through  December  1981.  This  was  a  retrospective  analy¬ 


sis. 


20 


With  respect  to  cost  per  closed  case,  all  cost  information  around 
purchased  assessments,  restoration,  training,  tools  and  equipment  is 
available  from  standard  financial  accounts.  The  cost  of  Rehabilitation 
Counsellor  time  for  each  closed  case,  to  be  added  to  the  purchased 
services  cost,  was  estimated  from  individual  worker's  estimates  (re¬ 
quested  periodically)  of  percentage  of  time  in  a  simple  week  spent  on 
direct  service,  administration,  and  public  relations  or  resource  devel¬ 
opment.  To  estimate  a  fair  share  of  General  Administration  time  and 
cost  for  each  closed  case,  aggregate  administration  cost  per  month  ms 
divided  by  the  average  number  of  active  eases  per  month  (September 
through  December  1981)  to  get  an  average  administrative  cost  per  active 
case  per  month.  This  monthly  cost  was  multiplied  by  the  number  of 
months  each  closed  case  was  in  active  status. 

With  respect  to  outcome  measurement,  the  Ministry's  Rehabilitation 
Closure  Report,  Form  70-00-022  (Nov. 2/76)  requires  that  each  Rehabilita¬ 
tion  Counsellor,  when  the  case  is  closed,  fill  out  a  "Status  At  Closure" 
section  by  checking  one  of  the  following  categorical  options:  []  Self- 

Employed,  [)  Sheltered  Employment,  []  Home-bound  Employment,  or  () 
Housewife  or  Homemaker.  One  may  add  an  additional  category:  []  Unem- 
pl oyed . 

For  scaling  purposes,  there  is  a  problem  if  one  attempts  to  treat 
"Housewife  or  Homemaker"  as  one  option  for  paid  employment. 

If  we  remove  "Housewife  or  Homemaker"  from  the  paid  employment 
list,  we  can  note  that  there  is  an  underlying  continuum  behind  the  cate¬ 
gories  running  from  greatest  to  least  autonomy.  However,  it  should  be 
observed  that  the  Vocational  Rehabilitation  Services  Act  and  Regulations 
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do  specify  that  improved  success  as  a  "Housewife  or  Homemaker"  is  a 
desirable  goal  in  itself  for  persons  not  capable  of  contemplating  paid 
employment.  However,  for  the  particular  analysis  to  follow,  we  will  set 
this  category  aside.  For  this  particular  exercise  in  cost-outcome  an¬ 
alysis,  we  have  consolidated  the  categories  into  a  three  point  ordinal 
scale  as  follows: 

A  High  Success  -  regular  employment  or  self-employed 

B  Medium  Success  -  sheltered  or  homebound  employment 

C  Low-No  Success  -  unemployed  (if  the  client  withdrew  from 

service,  this  is  also  defined  as  low-no.) 

Among  the  77  closed  cases  examined,  5  were  closed  with  a  special 
status  of  "independent  student"  and  1  was  closed  with  the  special  status 
of  "hospitalized".  These  6  cases  plus  3  cases  closed  as  ineligible  for 
service,  have  been  removed  from  the  analysis,  leaving  68  cases. 

Each  Rehabilitation  Closure  Report  records  "Disability-Cause  Diag¬ 
nosis".  The  problem  types  for  the  68  cases  can  be  catalogued  as  in 
Table  II-1. 

Insert  Table  II-1  here 

The  Rehabilitation  Closure  Report  also  includes  a  section  on 
"Services  Provided";  the  following  options  are  listed:  []  assessment, 
[]  restoration,  []  training,  (]  tools  and  equipment,  and  []  other. 

For  a  cost-outcome  analysis,  there  may  be  utility  in  baking  a  prob¬ 
lem-entity  approach  to  the  analysis  rather  than  an  overall  program  or 
all-probl ems-together  approach.  Kith  this  in  mind,  our  demonstration 
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TABLE  II-1 

DIAGNOSED  PROBLEMS  FOR  SIXTY-EIGHT 
VOCATIONAL  REHABILITATION  CLIENTS 


Probl cm 


Frequency 


Percentage 


#1 

Learning  disability 

10 

#2 

Sacroilias  backache 

2 

#3 

Bl  i  Iness 

1 

#4 

Alcohol  ism 

16 

#5 

Friedreich ' s  Ataxia 

1 

#6 

Pa  rapl  eg  ia 

2 

#7 

Cerebral  Palsy 

3 

#8 

Mental  retardation 

9 

#9 

Menieres  syndrome 

1 

#10 

Amputation  or  limb  damage 

3 

#11 

Kidney  failure 

1 

#12 

Congenital  Reverse  Kyphosis 

1 

#13 

Epil epsy 

3 

#  14 

Multiple  Sclerosis 

3 

#15 

Neoplasm  of  Esophagus 

1 

#  16 

Congenital  Nystagmus 

1 

#17 

Guillain  Barre  Syndrome 

1 

#  18 

Asthma 

1 

#19 

Arthritis 

3 

#20 

Muscular  dystrophy 

1 

#21 

Schizophrenia 

3 

#22 

Emphysema 

1 

14.7 
2.9 

1.5 

23.5 

1 . 5 
2.9 

4.4 
13.2 

1.5 

4.4 
1  . 

1  . 

4.4 

4.4 

1.5 

1.5 

1.5 

1.5 

4.4 

1.5 

4.4 

1.5 


Total 


68 


100.0 


Orthopaedic:  #2.  #5,  #6,  #7,  #10,  #12,  #14,  #17,  #19,  #20 

Mental  Retardation:  #8 


Alcoholism:  #4 


to  in 
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exercise  will  focus  on  three  problem  areas:  orthopaedic  problems  (#2, 

#5,  #6,  #7,  #10,  #12,  #14,  #17,  #19,  #20),  mental  retardation  (#0),  and 
alcoholism  (#4).  The  first  is  predominantly  a  biological  problem  area 
while  the  second  and  third  reflect  a  difficult  mix  of  both  biological 
and  social-cultural  factors. 

Table  II-2  presents  the  raw  data  for  20  orthopaedic  cases  including 
cost  and  rated  level  of  success  for  each  case. 

Insert  Table  II-2  here 

From  Table  II-3  we  see  that  59%  of  the  seventeen  orthopaedic  cases 
closed  between  October  and  December  1931  achieved  a  high  success  level 
or  outcome.  Eighty  percent  of  total  expenditures  was  invested  in  these 
cases.  Also  35%  of  cases  were  closed  at  a  low-no  success  level  with  10% 
of  total  expenditures  deployed. 

Insert  Table  II-3  here 

Table  II-4  and  II-5  are  similar  to  Tables  II-2  and  II-3  but  in  this 
case  are  for  the  M-R  cases.  From  Table  II-5,  it  is  observed  that  the 
proportion  of  cases  to  expenditures  at  each  level  of  success  is  fairly 
even . 


Insert  Tables  II-4  and  II-5  here 

In  Tables  II-6  and  II-7  the  same  analysis  is  repeated  for  the  alco¬ 
holism  cases.  Here,  the  proportion  spent  on  successful  cases  ras  greater 
than  the  number  of  cases  whereas  the  proportion  spent  on  low-no  success 
cases  was  less  than  the  proportion  of  cases.  Two  questions  arise:  would 
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TABLE  II-2 

ORTHOPAEDIC  CASES:  COST  AND  OUTCOME 


A  -  High  success  (regular  employment  or  self-employed) 
B  -  Medium  success  (sheltered  or  homebound  employment) 
C  -  Lo-no  success  (unemployed  or  withdrew) 


Case 

No . 

Days 

Open 

Purcha  sed 

($) 

Rehabil  ita- 

tion  Costs 
( Direct ) 

($) 

Rehabil  ita- 

tion  Costs 
(public  re¬ 
lations  or 

resource  d 
dev el opment ) 

($) 

Adninis- 

tra  t  ion 

($) 

Total 

($) 

Out¬ 

come 

2 

219 

143 

955 

77 

751 

1,926 

C 

7 

182 

150 

794 

64 

624 

1,632 

c 

10 

229 

305 

998 

80 

785 

2,  168 

c 

15 

1564 

1  1,079 

6,819 

547 

5,365 

23,810 

* 

16 

55 

150 

240 

19 

109 

593 

A 

17 

47 

143 

205 

16 

161 

525 

A 

23 

238 

2,020 

1,038 

83 

816 

3,957 

A 

26 

36 

150 

157 

13 

123 

443 

C 

33 

355 

2,520 

1,548 

124 

1,218 

5,410 

H 

36 

489 

5,335 

2,  132 

171 

1,677 

9,315 

B 

39 

342 

— 

1,491 

120 

1  ,  173 

2,784 

C 

50 

1001 

10,002 

4,364 

350 

3,433 

18, 149 

A 

51 

99 

156 

432 

35 

34  0 

963 

C 

55 

476 

2,755 

2,075 

167 

1,633 

6,630 

A 

63 

338 

156 

1,474 

118 

1 ,  159 

2,907 

A 

64 

392 

. 2,296 

1 , 709 

137 

1,345 

5,487 

A 

67 

287 

588 

1,251 

100 

984 

2,923 

A 

68 

279 

10,725 

1,216 

98 

9  57 

12,996 

H 

69 

59 

3,783 

257 

207 

202 

4,449 

H 

70 

671 

5,224 

2,926 

235 

2,302 

10,687 

A 

Total 

7358 

57,680 

32,081 

2,761 

25,237 

1  17,759 

Average  || 

368 

2,834 

1,604 

138 

1,262 

5,888 

25 


TABLE  II-3 

ORTHOPAEDIC  CASES:  COST-OUTCOME  ANALYSIS 


Note:  The  three  housewife  cases  are  not  classified  in  this  table 
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TABLE  II-4 

M-R  CASES:  COST  AND  OUTCOME 


A  -  High  success  (regular  employment  or  self-employed) 
B  -  Medium  success  (sheltered  or  homebound  employment) 
C  -  Lo-no  success  (unemployed  or  withdrew) 


Case 

No. 

toys 

Open 

Pur cha  sed 

($) 

Rehabil  ita- 

tion  Costs 
( Direct ) 

($) 

Rehabil  ita- 
tion  Costs 
(public  re¬ 
lations  or 

resource  d 
devel opment ) 
($) 

Adminis¬ 

tration 

($) 

Total 

($) 

Out- 

Come 

18 

61 

423 

266 

21 

209 

919 

A 

28 

616 

1,500 

2,686 

215 

2,  113 

6,514 

B 

29 

890 

981 

3,880 

312 

3,053 

8,226 

B 

30 

613 

1,358 

2,673 

215 

2,  103 

6,349 

B 

37 

668 

300 

2,912 

234 

2,291 

5,737 

B 

41 

246 

1,078 

1,072 

86 

844 

3,081 

B 

43 

753 

7,396 

3,283 

264 

2,583 

13,526 

C 

59 

64 

280 

279 

22 

220 

801 

C 

77 

19 

194 

83 

7 

65 

349 

C 

Total 

3930 

13,501 

17,134 

1,376 

13,481 

45,501 

Average 

437 

1,501 

1,904 

153 

1,498 

5,056 

TABLE  II-5 

M-R  CASES:  COST-OUTCOME  ANALYSIS 


High  Success 
lise  No.  Cost 

1  ($) 

Medium  Success 

Case  No.  Cost 

($) 

Low- no 

Case  No. 

Success 

Cost 

($) 

18  919 

28 

6,514 

43 

13,526 

29 

8, 226 

59 

801 

30 

6,  349 

77 

349 

37 

5,737 

in _ 

41 

3,081 

1 

otal  919 

Total 

29,907 

Total 

14,676 

||erage  919 

Average 

5,931 

Average 

4,892 

|i  2%  (expenditures) 

66% 

( expenditures ) 

32% 

(expenditures ) 

*11%  (cases) 

56% 

(ca  ses ) 

33% 

(cases ) 

ha  v  e  i  tu¬ 


mor  e  expenditure  on  those  cases  in  the  low-no  success  category 
proved  the  success  rate  or  were  the  counsellors  astute  enough  to  close 
cases  before  the  expenditure  per  case  became  unrealistic? 

Insert  Tables  II-6  and  II-7  here 


The  Utility  of  Such  Findings 

For  each  of  the  three  problem  entities,  high  or  medium  success  ms 
achieved  with  from  67%  to  65%  of  the  cases.  This  represents  solid 
achievement  and  can  be  a  source  of  some  reassurance  to  all  concerned.  It 
can  also  provide  a  benchmark  for  reference  in  subsequent  quarters  or  from 
year  to  year. 

On  the  other  hand,  from  35%  to  33%  achieved  no  real  progress.  Fur¬ 
ther,  the  case  number  (disguised  here)  of  those  cases  which  did  not  im¬ 
prove  is  known.  In  hospital  settings,  special  chart  studies  might  be 
done  to  try  to  find  out  what  if  any  characerist ics  such  patients  had  in 
common,  or,  what  in  the  treatment  approach  might  be  changed.  The  same 
kind  of  a  special  case  file  study  could  be  done  around  vocational  re¬ 
habilitation  cases  with  low-no  success. 

The  average  cost  per  successful  closed  case  was  $7,567.00  for  ortho- 

« 

paedic  cases,  $919.00  for  the  one  high  success  mental  retardation  case, 
and  $2,74  1.00  for  alcoholism  cases.  While  one  would  not  wish  to  depend 
on  such  figures  from  a  single  three  month  time  period,  over  a  period  of 
time,  replications  of  this  analysis  would  begin  to  provide  some  pretty 
dependable  estimates  on  cost  per  successful  closed  case.  Additional  spe¬ 
cial  case  file  studies  of  the  high  success  cases  could  give  some  indica- 


TABLE  II-6 


ALCOHOLISM  CASES:  COST  AND  OUTCOME 


A  -  High  Success  (regular  employment  or  self-employed) 
B  -  Medium  success  (sheltered  or  homebound  employment) 
C  -  Low- no  success  (unemployed  or  withdrew) 


Case 

No . 

Days 

Open 

Purcha  sed 

($) 

Rehab il  ita- 

tion  Costs 
( Direct ) 

($) 

Rehab  il  ita- 

tion  Costs 
(public  re¬ 
lations  or 

resource  d 
devel opment ) 

($) 

Adni nis- 

tra  tion 

($) 

Total 

($) 

Out¬ 

come 

8 

1  18 

12 

514 

41 

405 

972 

C 

9 

155 

18 

676 

54 

532 

1 , 280 

c 

11 

34 

18 

148 

12 

117 

295 

A 

12 

71 

50 

310 

25 

244 

629 

A 

13 

102 

18 

445 

36 

350 

849 

C 

14 

247 

- 

1,077 

86 

847 

2,010 

C 

46 

300 

1,044 

1,308 

105 

1,029 

3,486 

A 

47 

206 

269 

898 

72 

707 

1,946 

A 

48 

126 

110 

549 

44 

432 

1,135 

A 

49 

29 

12 

126 

10 

99 

247 

C 

53 

150 

179 

654 

53 

515 

1,401 

A 

57 

62 

150 

270 

22 

213 

655 

C 

72 

293 

524 

1,277 

103 

1,005 

2,909 

A 

73 

286 

107 

1,247 

100 

931 

2,435 

A 

74 

711 

5,600 

3,100 

249 

2,439 

11,388 

A 

75 

193 

214 

841 

68 

662 

1,785 

A 

Total 

3083 

8,325 

13,440 

1,080 

10,577 

33,422 

Average 

193 

— 

520 

840 

- -  -  -  - - 

68 

661 

2,089 

30 


TABLE  II-7 

ALCOHOLISM  CASES:  COST-OUTCOME  ANALYSIS 
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tion  of  the  kinds  of  cases  for  which  high  success  at  a  certain  cost  could 
be  predicted. 

Finally,  information  is  produced  from  which  to  decide  whether  to 
retain  programs  for  certain  problem  entities,  or  to  design  a  planned 
alternative  program  for  certain  problem  entities  on  which  comparative 
cost-outcome  results  will  then  be  calculated.  A  randomized  allocation 
of  clients  to  the  two  programs  (on-going  and  planned  alternative)  would 
be  desirable  and  for  the  comparison  period,  closer  monitoring  of  treat¬ 
ment  activity  might  be  necessary. 

Perhaps  che  greatest  single  weakness  in  this  feasibility  demonstra¬ 
tion  has  been  the  reliance  on  average  time  and  cost  figures  for  all  act¬ 
ive  cases  in  one  month  (October  1931).  It  is  likely  that  the  amount  of 
Rehabilitation  Counsellor  time  lias  varied  considerably  from  the  average 
for  individual  cases.  To  capture  direct  service  time  more  accurately  for 
each  case  would  require  the  use  of  a  daily  worker  time  form  to  be  com¬ 
pleted  each  day's  end  and  dropped  in  the  out  tray. 

HESPELER  PRE-SCHOOL  EDUCATION  CENTRE 

The  Hespeler  Pre-school  Education  Centre,  located  at  26  Cooper 
Street,  Cambridge  (Hespeler),  Ontario,  is  one  of  five  da  y  nurseries  ad¬ 
ministered  directly  by  the  Department  of  Health  and  Social  Services,  Reg¬ 
ional  Municipality  of  Waterloo.  In  addition  to  this  network  of  munici¬ 
pally  administered  d*y  are  centres,  there  were  forty-four  voluntary 
nonprofit  and  twenty-six  profit  oriented  nursery  schools  and  diy  care 
centres  serving  the  Regional  Waterloo  population  area  in  1982. 
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The  Hespeler  Centre  serves  seventy  pre-school  children  of  whom  about 
60%  are  children  with  no  problems  and  about  40%  are  special  needs  child¬ 
ren  who  are  suffering  from  varying  degrees  of  developmental  deficit.  None 
of  the  children  are  mentally  handicapped.  The  Centre,  then,  illustrates 
the  normalization  approach  to  remedial  education  with  pre-school  children 
and  the  use  of  relatively  normal  children  as  role  models  and  sources  of 
peer  influence  for  other  children  with  special  needs. 

The  Hespeler  Centre,  however,  does  not  simply  rely  on  the  contribu¬ 
tion  of  normal  children  within  a  growth  inducing  milleau,  but  provides 
highly  individualized  remedial  education  and  training  attention  as  well. 
Prior  to  September  1981,  the  Denver  Developmental  Screening  Test,  the 
Portage  Test,  and  other  screening  tests  were  used  to  identify  the  child¬ 
ren  who  might  be  in  deficit  and  to  pin-point  areas  of  deficit.  Such  in¬ 
formation  is  used  to  guide  the  selection  and  use  of  specific  remedial 
routines  with  individual  children  around  small  motor  skills,  social 
skills,  expressive  language  ability,  etc.  Udaen  indicated,  special  diag¬ 
nostic  studies  for  individual  children  are  arranged,  particularly  when 
the  remedial  work  routines  do  not  lead  fairly  promptly  to  improved  func¬ 
tioning  . 

In  the  Hespeler  Pre-school  Education  Centre,  some  of  the  seventy- 

« 

three  children  attend  morning  sessions,  the  remainder  attending  in  the 
afternoon.  The  special  needs  children  attend  five  half-days  a  week,  and 
the  regular  children  just  three  half-days  a  week,  with  an  occasional 
exception . 

Beginning  in  September  1981,  the  Hespeler  Centre  began  the  installa¬ 


tion  and  use  of  a  new  screening  instrument,  the  Developmental  Inventory 
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Scales  for  Children  (DISC)  supplementing  the  Denver  Developmental  Screen¬ 
ing  Test . * 

The  DISC  is  a  primary  screening  test  to  detect  developmental  prob¬ 
lems  in  infants  and  preschoolers,  and  covers  the  age  range  of  birth 
through  five  years  (60  months).  Eight  developmental  areas  are  assessed: 
fine  motor,  gross  motor,  receptive  language,  expressive  language,  visual 
attention  and  memory,  auditory  attention  and  memory,  self-help  and  soci.il 
skills. 

For  each  skill  area  the  range  of  possible  scores  is  from  1  to  27. 
Also,  for  each  skill  area,  percentiles  have  been  established  for  a  norma¬ 
tive  sample  of  children  at  eleven  age  levels:  0-3  months,  4-6  months,  7- 
12  months,  13-18  months,  19-24  months,  25-30  months,  31-36  months,  37-42 
months,  43-43  months,  49-54  months,  and  55-60  months.  An  illustration  for 
one  of  the  six  skill  areas,  fine  motor  skills,  is  reproduced  in  Table 
III-1 . 

Insert  Table  III-1  here 

A  fine  motor  skills  score  of  17,  for  example,  was  the  best  that  any 
of  the  13-18  month  old  or  19-24  month  old  normal  children  obtained.  At  a 
score  of  17,  100.00%  of  the  children  are  accounted  for.  However,  among 

the  25-30  month  old  children,  64.17%  got  a  score  of  17  or  less  which 

« 

means,  conversely  that  35.71%  of  these  normal  children  got  a  score  of  18 
or  higher;  indeed,  at  least  a  few  children  got  a  score  as  high  as  23, 
among  the  25-30  month  old  children  used  to  age  norm  this  instrument. 

*  This  new,  standardized  developmental  screening  instrument  liad  just 
been  completed  by  Jeanette  Amdur,  Ph.D.,  Psychologist  and  Co-ordinator  of 
the  Child  and  Family  Centre,  Kitchener-Waterloo  Hospital,  with  the  col¬ 
laboration  of  Marion  Mainland,  M.A.  ( Psychol ogy ) . 
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TABLE  III-1 

DEVELOPMENTAL  INVENTORY  SCALES  FOR  CHILDREN 

FINE  MOTOR  ~ 

Percentage  of  Children  in  Normative  Sample 


Obta ined 


Age  in  Months 
Score  0-3  4-6  7-12  13-18  19-24  25-30 


31-36  37-42  43-48  49-54 


1  2.08 

2  12.50 

3  72.58 

4  100.00  46.15 

5  71.79 


6 

84.62  3.92 

7 

94.87  41.18 

8 

100.00  74.51 

9 

88.23 

6.38 

10 

96.08 

12.77 

1 1 

96.08 

25.53 

2.17 

12 

100.00 

74.47 

6.52 

13 

91.49 

47.83 

14 

93.62 

67.39 

15 

95.75 

89.13 

16 

97.87 

93.48 

17 

100.00 

100.00 

18 

19 

20 
21 
22 

23 

24 

25 

26 


14.29 

23.81  6.78 
42.86  8.48 

64.29  11.86 

76.19  22.03 


83.33 

30.51 

3.33 

92.86 

38.98 

10 . 00 

1.61 

97.62 

67.80 

13.33 

3.23 

83.05 

26.67 

4.84 

100.00 

91.53 

55.00 

8.07 

94.92 

70.00 

29.03 

94.92 

96 . 67 

64.52 

7.27 

100.00 

90.67 

83.87 

25.45 
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The  developers  of  the  DISC  instrument  have  defined  the  75th  per¬ 
centile  for  a  give  age  range  as  reflecting  a  "normal"  level  of  function¬ 
ing.  Among  25-30  month  old  children,  then,  a  DISC  score  of  18  or  higher 
designates  a  normal  level  of  functioning  (percentile  standing  of  76.19%). 
A  child  with  a  percentile  standing  of  60-74%  is  classified  as  "question¬ 
able";  a  standing  of  50-59%  is  classified  as  "borderline  abnormal";  a 
standing  of  0-50%  is  "abnormal". 

Table  III-2  below  reports  the  DISC  scores  and  percentile  standing 
(PS)  in  October  1981  and  after  five  months  of  remedial  work  in  April  1982 
for  one  male  child  (7M)  covering  six  of  the  eight  skill  areas.  Ratings 
for  the  remaining  two  skill  areas  could  not  be  obtained  in  October  1982 
due  to  test  refusal  or  absence. 

Insert  Ihble  III-2  here 

The  absence  of  complete  data  on  all  eight  skill  areas  is  not  vital 
here  since  we  are  seeking  only  to  demonstrate  how  the  DISC  instrument  and 
its  percentile  tables  can  be  used  to  derive  a  single  overall  score  for  a 
child  at  two  points  in  time.  In  Table  III-2,  we  lrave  averaged  the  per¬ 
centile  standings  across  all  six  skill  areas  for  which  we  laave  both 
October  and  April  data.  The  average  percentile  standing  level  for  7M  is 
found  to  be  1.21%  in  October,  16.20%  in  April  with  an  improvement  or 
change  in  standing  of  14.99  percentile  points.  The  plus  sign  for  the 
three  skill  areas  in  which  the  child's  percentile  standing  was  0.00  in 
October  1981  reminds  us  that  the  actual  improvement  in  those  skill  areas 
may  be  something  greater  than  the  difference  reported  since  the  instru¬ 
ment  does  not  make  provision  for  negative  scores  below  0.00. 


TABLE  II I~2 


RAW  SCORES  AND  PERCENTILE  STANDING  FOR  ONE  CHILD  ( 7M ) 
IN  SIX  SKILL  AREAS,  OCTOBER  1981  AND  APRIL  1982 


October  1981 

April 

1982 

Improvement  in 

Skill 

Percentile 

PS  *  :  54  months 

^ : 59  months 

Area 

Score 

Score 

PS 

Standing  (IPS) 

Fine  Motor  (FM) 

25 

7.27 

26 

53.32 

46.25 

Receptive 
Language  (  RL) 

22 

0 . 00 

24 

12.68 

12.68 

Expressive 
Language  (EL) 

20 

0.00 

22 

0.00 

0.00 

Gross  Motor 

(GM) 

22 

0.00 

23 

26.76 

26.76 

Auditory- Atten¬ 
tion/Memory 

21 

0.00 

23 

0.00 

0.00 

Visual -Atten- 
t  ion/l-Iemory 

21 

0.00 

24 

4.23 

4.23 

Self-Help  (SH) 

- 

- 

Social  Skills 
(SS) 

- 

- 

Average 

1.21 

16.20 

14 .99 

*  PS  -  Percentile  Standing.  Normal:  75%  up.;  Question.abl  e :  60-74%; 

Borderline  Abnormal  :  50-59%;  Abnornaal  :  0-49% 

^  IPS  -  Improvement  in  Percentile  Standing. 
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Also  note  that  in  a  developmental  area  such  as  expressive  language 
in  which  child  7M  was  at  0.00  percentile  standing  in  both  October  and 
April,  the  absence  of  negative  scores  in  the  percentile  table  nukes  it 
impossible  to  credit  some  progress  which  was  actually  nude;  this  child's 
test  score  for  expressive  language  did  improve  from  20  to  22,  but  the 
amount  of  improvement  still  did  not  bring  the  child  above  the  0.00  per¬ 
cent  il  e  standing.  Use  of  the  DISC  instrument  and  its  percentile  tables 
does  report  improvement  in  each  developmental  area  in  which  a  child  is  at 
least  a  little  above  0.00  percentile  standing,  but  not  for  a  child  who 
begins  and  ends  at  some  negative  percentile  level  below  0.00  percentile 
standing.  This  inevitable  limitation  in  any  standardized  developmental 
inventory  instrument  can  be  compensated  informally  by  a  separa te  look  at 
raw  score  improvements  below  the  0.00  percentile  standing  level. ^ 

It  should  be  noted  that  the  basic  purpose  of  the  DISC  instrument  as 
a  developmental  screen  is  to  uncover  the  specific  skill  areas  in  which  a 
child  may  be  in  need  of  remedial  work.  Study  child  7M  was  at  borderline 
abnormal  in  fine  motor  skills  (53.52  PS)  at  the  end  of  the  five  month 
treatment  period,  but  remained  very  low  in  PS  on  the  other  five  skill 
areas.  To  average  this  child's  PS  percentages  to  get  a  single  value  on 
overall  functioning  level  (a  mean  PS  in  April  1982  if  16.20%)  over  six 
skill  areas  has  no  usefulness  at  all  to  the  early  childhood  education 
specialists  working  with  these  children  from  day  to  day  at  the  Hespeler 
Pre-School  Education  Centre.  However,  for  cost-out  come  analysis  purposes, 

2 

A  brief  explanation  of  a  further  option  for  outcome  measurement 
based  on  chronological  age  gain  in  months  will  be  separately  illustrated 
at  the  close  of  this  discussion. 
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six  or  eight  separate  percentile  standing  ratings  for  a  given  child  are 
equally  useless.  A  single  overall  rating  across  all  skill  areas  is 
essential . 

In  Table  III-3  below  we  present,  for  each  of  the  six  study  children, 
the  overall,  average  PS  ratings  for  October  1991,  April  1982,  and  the 
consequent  improvement  in  percentile  standing  for  each  child  (IPS). 

Insert  Table  III-3  here 

All  six  study  children  classify  as  of  October  1981  as  abnormal  in 
the  depth  of  their  developmental  deficits.  While  improvement  is  apparent 
at  the  end  of  five  months  of  remedial  work,  with  the  exception  of  child 
2M  who  registers  no  improvement  in  percentile  standing,  the  children  are 
still  functioning  at  an  abnormal  level,  i.e.,  below  50. 

Pre  and  post  DISC  scores  for  each  of  the  project  children  were 
obtained  as  of  November  1,  1981  and  March  31,  1982. 

It  was  agreed  that  the  new  DISC  screen  would  be  administered  to 
about  ten  special  needs  children  in  October  1991  and  again  in  April  1992, 
as  a  measure  of  treatment  outcome.  Eleven  special  needs  children  were 
initially  selected  for  the  study,  anticipating  some  loss  from  family 

moves.  Complete  data  for  six  children  are  reported  here. 

« 

It  was  further  agreed  that  a  Worker  Activity  and  Time  Report  Shoot 
would  be  completed  each  day  by  each  teacher  to  report  individual  or  small 
group  remedial  work  time  with  any  or  all  of  the  eleven  initial  project 
children.  The  plan  was  to  estimate  total  program  cost  for  each  of  the 
special  needs  children  by  adding  individual  remedial  work  time  to  the 
basic  cost  for  regular  and  special  children  in  general  milleau  activities 
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TABLE  II 1-3 

AVERAGE  PERCENTILE  STANDING  IN  OCTOBER  1981  AND  APRIL  1982 
OVER  SIX  SKILL  AREAS  FOR  SIX  STUDY  CHILDREN; 
HESPELER  PRE-SCHOOL  EDUCATION  CENTRE 


Child 

PS  1 

October  1981 

PS  1 

April  1982 

IPS  2 

2M 

0 . 00 

0.00 

0.00 

3M 

11.50 

28.04 

16.54 

5F 

0.00 

20.66 

20.66 

7M 

1.21 

16.20 

14.99 

9F 

4.85 

19. 17 

14.32 

1  IF 

1 .21 

26.29 

25.08 

*  PS  -  Percentile  Standing.  Normal :  75%  up.;  Questionable;  60-74%; 

Borderline  Abnormal:  50-59%;  Abnormal:  0-49% 

O 

IPS  -  Improvement  in  Percentile  Standing. 
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In  October  1981/  the  first  DISC  ratings  for  the  study  children  were 
prepared,  following  a  training  period  conducted  by  a  DISC  trainer  using  a 
video-tape  demonstration.  Two  of  the  eleven  children  moved  away.  Of  the 
remaining  nine  children,  absence  or  test  refusal  elimimted  three.  Com¬ 
plete  data  were  obtained  on  six.  The  findings  from  the  Hespeler  Pre¬ 
school  Education  Centre  cost-outcome  feasibility  demonstration  are  pre¬ 
sented  in  the  following  five  tables. 

Table  III-4  presents  the  cost  for  each  study  child  for  the  period 
November  1,  1981  to  March  31  ,  1982.  We  see  that  the  average  cost  per 

study  child  for  the  five  month  period  was  $1258  as  compared  to  $651  for  a 
regular  child.  The  difference  in  cost  is  a  reflection  of  two  factors: 
the  special  needs  children  attend  five  lialf  days  a  week  instead  of  three 
half  days  as  for  the  regular  children,  and  the  special  needs  children 
receive  individualized  remedial  attention  from  individual  teachers,  which 
the  regular  children  do  not. 

Insert  Table  III-4  here 

It  must  be  noted  that  since  the  two  implicit  "programs"  (one  for 
regular  children  and  one  for  special  needs  children)  are  just  one  program 
at  an  operational  level,  no  separate  financial  accounting  is  possible  for 
the  two  groups  of  children.  As  a  result,  estimates  of  separate  costs 
must  be  made  as  in  our  illustration. 

It  can  be  noted  here  that  there  were  844  hours  of  staff  volunteer 
time  (including  students)  contributed  to  the  Centre  program  during  the 
five  month  period.  These  844  volunteer  hours  represent  19%  of  the  total 
unpaid  volunteer  and  paid  staff  hours  (4441  hours)  for  the  study  period . 


TABLE  II 1-4 


COST  FOR  SINGLE  CASES:  HESPELER  PRE-SCHOOL  EDUCATION  CENTRE 


(For  a  treatment  period  of  five  months, 
November  1,  1981  to  March  31,  1982) 


Child 

Maximum 

Centre 
Hours  * 

Special 

Remedial 

y 

Hours 

Remedial 

Hours 

Cost 

($) 

Net 

M  i  1 1  ea  u 

Hours 

Mill  ea  u 

H  o  ur  s 
Cost  ^ 

($) 

Total  Cost 

Per  Special 

Ch  il  d 

($) 

2M 

322 

35.6 

548 

286.4 

859 

1407 

3M 

322 

40.6 

625 

281.4 

844 

1469 

5F 

322 

31.9 

491 

290.1 

870 

1361 

7M 

322 

10.9 

168 

3  11.1 

933 

1101 

9F 

322 

10.4 

160 

311.6 

935 

1095 

1  IF 

322 

11.9 

183 

310.1 

930 

1113 

Total 

141.3 

2175 

1790.7 

5371 

7546 

Average 

23.6 

362.5 

298.5 

S95.2 

1257.7 

Notes : 

*  There  were 

92  week  days 

in  the  five  month 

period.  A 

special  needs 

child 

attends  5 

half-days  a 

week  (3.5 

hours  of 

program  time  each  day). 

92  x  3 

.5  yields 

322  hours  as 

the  maximum  available  hours 

for  a  special 

needs 

child.  A 

regular  child  attends 

three  half-days  in 

the  five  month 

period 

.  55.2  x 

3.5  yields  194.25  hours  as  the 

maximum  available  for  a 

regular  child. 

The  average  enrollment  over  the  five  treatment  months  was  29  specia 
needs  children  and  44  regular  children.  The  total  potential  child  hours 
therefore,  were  as  follows: 

29  special  children:  29  x  322  =  9338  child  hours 

44  regular  children:  44  x  194.25  =  8547  child  hours 


Total  child  hours  (100%) 


17885  child  hours 
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TABLE  II 1-4  (cont'd) 

^  From  special  time  sheets  completed  by  each  early  childhood  education 
worker  at  the  centre,  individual  remedial  work  hours  per  child  are  known 
over  the  five  month  period.  The  total  was  141.3  remedial  work  hours. 

The  total  input  cost  at  the  Centre  for  the  five  months  was  $55,408. 
Over  92  days  this  amounts  to  $301.57  per  half  day  or  $36.16  per  hour 
(half-day  =  3.5  hours).  There  were  5.6  full  time  position  equivalents. 
$86.16  t  5.6  =  $15.39  as  the  average  position  cost  per  input  hour  (not 
salary  cost  alone). 

The  individualized  remedial  work,  then,  had  a  cost  of  $15.39  per 
hour.  For  the  141.3  remedial  hours  the  total  cost  was  $2175.  The  av¬ 
erage  remedial  work  cost  per  study  child  was  $363.  The  average  total 
cost  per  study  child  was  $1258. 

The  cost  figures  are  based  upon  maximum  possible  attendance  hours 
and  not  upon  the  actual  attendance  hours  for  each  child.  The  cost  re¬ 
mains  the  same  whether  the  child  is  present  or  not.  To  estimate  milieu 
cost,  first  calculate  the  net  number  of  milieu  hours: 

Total  child  hours  17,885.0 

Remedial  work  hours  141.3 


Net  milieu  hours  17,743.7  (or  17,744) 

If  $2175  was  spent  for  individualized  remedial  work  with  the  six 
study  children,  the  net  input  expenditure  available  for  milieu  activity 
could  be: 

Total  input  cost  $  55,488 

Remedial  work  cost  2,175 


Net  milieu  cost  $  53,313 

A  total  of  17,444  child  hours  were  available  for  milieu  activity  at 
a  cost  of  $53,313.  $53,313  *  17,744  =  $3.00  as  the  average  cost  per 

child  (regular  or  special)  per  milieu  hour. 

The  44  regular  children  vised  9547  child  hours  as  noted  earlier. 
9547  hours  x  $3.00  =  $28,641  spent  for  the  44  regular  children.  $28,641 
v  44  =  $651  as  the  average  cost  for  each  regular  child  over  the  five 
month  period. 

For  the  five  month  period,  then,  the  average  cost  for  each  special 
child  was  $1258  and  for  each  regular  child  was  $651. 
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The  actual  salaries  and  fringe  benefit  costs  for  the  period  were  $44,693. 
We  can,  therefore,  estimate  a  savings  of  about  $10,483  made  possible  by 
the  contribution  of  labor  by  staff  volunteers.  The  total  personnel  cost 
would  have  been  about  $55,  175  had  not  the  volunteers  been  there. 

In  Table  III-5,  the  cost  for  each  child  in  relation  to  outcome  is 
displayed . 

Insert  Table  III-5  here 

Following  upon  Table  III-5,  the  question  arises  as  to  how  much  money 
might  be  involved  to  bring  each  of  the  six  children  up  to  a  normal  level 
of  functioning.  A  major  assumption  here,  of  course,  would  be  that  the 
rate  of  improvement  would  hold  reasonably  steady.  Such  an  assumption 
could  only  be  tested  by  a  longitudinal  study  for  the  estimated  additional 
months  of  remedial  work  implicit  in  the  estimated  total  cost  figure. 

Insert  Table  III-6  here 

Since  the  use  of  on-going  information  system  data  on  cost  and  out¬ 
come,  aggregated  from  single  subjects,  has  been  demonstrated  here,  there 
has  been  _no  sampling  or  randomized  allocation  procedure  used  as  would  be 
required  for  a  large,  one-shot  inferential  level  evaluative  research 
study.  When  aggregation  of  single  subject  data  is  the  preferred  research 
approach,  repl ication  of  findings  can,  for  internal  validity  proposes  but 
not  for  external  validity  purposes,  substantially  take  the  place  of  ran¬ 
domized  experimental  and  control  group  procedures. 

This  means  that  the  findings  for  the  period  November  1,  1931  to 
March  31,  1982  with  these  six  children  for  whom  complete  DISC  data  were 


TABLE  III-5 


COST  AND  OUTCOME:  SIX  SPECIAL  NEEDS  CHILDREN 


Child 

Cost 

($) 

Improvement 
in  Percentile 
Standing  (IPS) 

Cost  per  IPS 
Point 

($) 

2M 

1407 

0.00 

oo 

3M 

1469 

16.54 

S9 . 00 

5F 

1361 

20.66 

66.00 

7M 

1101 

14.99 

73.00 

9F 

1095 

14.32 

76.00 

1 1F 

1113 

25.08 

44 . 00 
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TABLE  II 1-6 

ESTIMATE  OF  TOTAL  COST  TO  BRING  EACH  CHILD 
UP  TO  NORMAL  FUNCTIONING  LEVEL 


Child 

Percentile 

Sta  nding 
April  1 982  * 

Percentil e 

Points  in 

Deficit 

Cost  per 

Percentile  Point 

($) 

Estimated  Total 

Cost  to  Normal 
Functioning 
($) 

2M 

0.00 

75.00 

CO 

OO 

3M 

28.04 

46.96 

89.00 

4179.00 

5F 

20.66 

54.34 

66.00 

3586.00 

7M 

16.20 

58.00 

73.00 

4234 .00 

9F 

19. 17 

55.83 

76.00 

4243.00 

1 1F 

26.29 

48.71 

44.00 

2143.00 

Notes : 


1 


Six 


skill  areas  only. 
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obtained  would  need  to  be  buttressed  by  replications  with  as  many  special 
needs  children  as  possible  from  year  to  year. 

If  this  were  to  be  done,  and  if  the  treatment  outcomes  and  costs  for 
special  needs  children  appeared  to  be  somewhat  similar  from  year  to  year, 
voters  and  taxpayers  in  the  Waterloo  Region  and  elected  officials  in  the 
Regional  Council  of  the  Regional  Municipality  of  Waterloo  could  begin  to 
face  the  clear  policy  question  of  what  priority  to  give  to  remedial  work 
with  special  needs  children  within  the  Regional  Waterloo  population  area. 
The  cost  for  achieving  a  given  amount  of  improvement  would  be  known  with¬ 
in  reasonable  limits.  The  only  other  planning  and  funding  question  would 
be:  how  many  special  needs  preschoolers  are  there  in  the  Regional 
Waterloo  population  area? 

The  Chronological  Months  Gain  Option  for  Outcome  Measurement 

Reference  was  made  earlier  to  one  alternative  approach  to  outcome 
measurement  using  chronological  age  in  months  (corresponding  to  each  raw 
score)  as  a  measure  of  treatment  outcome,  instead  of  improved  percentile 
standing  (IPS)  as  in  this  demonstration.  To  explain  this  option,  the 
DISC  Record  Sheet  is  reproduced  below.  The  27  "items"  are  each  discrete 

statements  of  a  skill  which  a  child,  when  tested,  cither  ean  or  cannot 

« 

do.  In  general,  the  chronological  age  levels  in  months  (on  the  right) 
represent  the  age  at  which  a  normal  child  can  be  expected  to  execute  the 


prescribed  test  skill  successfully. 
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D. I . S.C. 


Record  Sheet 


d.i .s.c. 

Raw  Score 


Item 

27 

26 

25 

24 

23 

22 

21 

20 

19 

18 

17 

16 

15 

14 

13 

12 

11 

10 

9 

a 

7 

6 

5 

4 

3 

2 

1 


rro  .  /  yr . 


53-60  5 

55-57 
52-54 
4  0-51 
46-48  4 

43-45 
40-42 
37-39 
34-36  3 

31-33 
28-30 
25-27 
23-24  2 

21-22 
19-20 
17-18 
1  S  -16 
13-14 
11-12 
9-10 
7-8 
5-6 
4 
3 
2 
1 
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In  liable  III-3,  child  2M  is  shown  to  hive  been  at  0.00  percential 
level  in  October  before  the  five  month  treatment  period,  and  at  the  0.00 
percentile  level  still  in  April  following,  thus  showing  no  improvement  in 
percentile  standing  (IPS).  In  Table  III-3,  only  the  average  percentile 
standing  across  six  developmental  areas  is  shown,  and  no  raw  scores  for 
each  of  the  six  areas  are  displayed.  We  will  now  present  the  raw  scores 
for  child  24  (somewhat  similar  to  the  format  for  child  714  in  Table  III-2). 

Insert  Table  III-7  here 

Note,  then,  that  child  24  who  shows  no  improvement  when  percentile 
standing  (PS)  is  used  as  a  measure  of  outcome,  shows  evidence  of  improve¬ 
ment  when  chronological  months  of  gain  (C4G)  is  used  to  measure  level  of 
functioning  and  treatment  outcome. 

The  CMG  evidence,  however,  must  be  interpreted  with  two  points  in 

mind : 


1.  Child  2M  grew  5  months  older  during  the  five  month  treatment  period. 
With  this  consideration  in  mind,  child  21  in  fine  motor  skills,  for 
example,  could  be  said  to  have  fallen  2  months  behind  with  only  a  3 
month  gain;  the  same  would  be  true  for  visual-attention  and  memory 
skills.  However,  after  subtracting  5  months  for  aging,  child  21 
could  still  be  seen  as  having  made  net  chronological  months  gains  of 
from  1  to  7  months  in  the  other  skill  areas.  His  overall  average 
gain  in  chronological  months  was  7,  or  2  over  his  5  for  aging. 

2.  The  second  limitation  in  the  GIG  approach  his  to  do,  of  course,  with 
the  normless  ness  of  the  C4G  ratings.  The  importance  of  the  per¬ 
centile  standing  (PS)  approach  ray  be  in  its  link  to  standardised 
norms  of  skill  level;  something  currently  referred  to  in  the  litera¬ 
ture  as  "change  norms". 
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TABLE  II 1-7 

RAW  SCORES  AND  CHRONOLOGICAL  MONTHS  OF  GAIN 
FOR  ONE  CHILD  ( 2M )  IN  SIX  SKILL  AREAS ,  OCTOBER  1981  AND  APRIL  1982 


Skill  Areas 

October  1981 

Raw  Score 

April  1982 
Raw  Score 

Chronological  Months 
of  Gn  in  (CMG)  1 

Fine  Motor  (FM) 

23 

24 

(median  month  in 
each  range) 

3  months 

Receptive  Language  (RL) 

20 

22 

6  months 

Expressive  Language  (EL) 

19 

23 

12  months 

Gross  Motor  (<24) 

16 

20 

12  months 

Audi tory-Attent ion/ 
Memory 

19 

20 

3  months 

Visual -Attention/ 

Memory 

20 

22 

6  months 

Self-Help  (SH) 

_  2 

25 

- 

Social  Skills  (SS) 

_  2 

25 

- 

X  =  7  months 

Notes : 

*  CMG  “  Chronological  Months  of  Gain 
2 

No  October  score  available  due  to  absence  or  test  refused. 
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Comparison  of  client  change  da  ta  to  relevant  change  norms 
is  conceptually  the  strongest  approach  to  demonstrating 
program  effectiveness.  At  present,  such  change  norms  do 
not  exist  for  most  service  programs.  In  our  experience, 
personnel  in  regulatory  and  funding  agencies  often  con¬ 
sider  the  development  of  outcome  norms  to  be  the  obvious 
next  step  in  human  service  program  eval ua tion . . . 

(Hargreaves  and  Attkisson,  1992,  313).  ^ 


Until  this  demonstration,  the  researchers  are  not  aware  of  any  ex¬ 
amples  of  the  use  of  standardized  developmentary  inventory  instruments  to 
obtain  a  single,  overall  rating  of  progress  or  change  in  treatment.  For 
this  reason,  this  field  exploration  of  possibilities  and  problems  in 
doing  so  has  special  utility,  despite  the  extra  hours  of  consultation  and 
reflection  required  by  all  parties  directly  involved  in  this  particular 
field  demonstration. 

In  closing,  note  that  professional  colleagues  closely  involved  with 
the  Hespeler  Preschool  Education  Centre  and  its  operation  are  aware  of 
other  factors,  such  as  family  functioning  in  the  child's  home  and  the 
depth  of  deficit  in  certain  crucial  developmental  areas.  Such  factors 
may  explain  why  one  child  progresses  more  or  less  than  another.  Special 
file  studies  of  children  with  certain  kinds  of  deficit  or  certain  home 
situations  could  be  useful  in  this  regard.  New  approaches  to  remedial 
work  may  arise  which  can"  be  tried.  Comparative  cost-outcome  findings  era  n 
be  examined  for  different  approaches  with  children  for  whom  family  fact¬ 
ors  or  special  deficit  conditions  exist.  Time  Ins  not  permitted  further 


^  From  William  A.  Hargreaves  and  Clifford  C.  Attkisson,  "Evaluating 
Program  Outcomes,"  in  C.  Clifford  Attkisson,  William  A.  Hargreaves,  and 
Hardi  J.  Horowitz  (eds.).  Evaluation  of  Human  Service  Programs  (New  York: 
Academic  Press,  1978). 
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examination  of  such  questions  in  this  demonstration.  The  purpose  here 
has  been  to  explore  the  problems  and  possibil it ies  for  installing  and 
using  cost-outcome  procedures  as  an  extension  of  the  Centre's  present 
financial  and  program  accounting  system. 

If  the  Cay  Care  Division  of  the  Department  of  Social  Services  of  the 
Regional  Municipality  of  Waterloo  were  to  choose  to  integrate  an  annual 
cost-outcome  analysis  for  each  and  all  special  needs  children  at  the 
Hespeler  Pre-School  Education  Centre  on  the  forma  t  demonstrated  here, 
just  four  additional  steps  beyond  what  is  already  being  done  would  seem 
to  be  required:  (1)  the  Daily  Worker  Activity  and  Time  Report  Sheet  used 

by  Hespeler  Centx'e  teachers  in  the  demonstration  to  record  remedial  work 
time  spent  with  each  special  needs  child  each  morning  and  each  afternoon 
would  have  to  be  mode  a  permanent  procedure  at  the  Centre,  (2)  the 
Regional  Government  Department  of  Social  Services  would  have  to  receive 
these  sheets  each  month  and  store  the  data  in  a  computerized  data  base, 
(3)  the  Finance  Department  of  Regional  Government  would  have  to  prepare 
Table  III-5  each  year  end,  and  (4)  the  Day  Care  Unit  would  have  to  pre¬ 
pare  Thble  III-4  using  DISC  outcome  data  available  in  each  child’s  file 
at  the  Centre;  the  Hespeler  Centre  supervisor  would  probably  be  given  a 
one-page  form  on  which  to  report  this  information  to  the  Cay  Care  Unit. 

AUSABLE  SPRINGS,  INC. 

« 

(Group  Home) 

Ausable  Springs,  Inc.  is  a  network  of  14  Group  Homes  at  various  loc¬ 
ations  in  Ontario.  With  about  6  beds  at  each  site,  some  84  beds  in  all 
are  involved.  This  agency  is  incorporated  as  a  non-proft  organization. 
It  has  a  self-perpetuating  Board  of  Directors  of  4  members.  The  time 
frame  for  da  ta  collection  as  set  at  September  through  December  1981. 
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Sixteen  cases  were  studied.  Four  were  closed  during  the  three  month  time 
frame  and  twelve  remained  open  cases.  The  outcome  rating  on  all  sixteen 
cases  was  done  as  of  December  31,  1981.  With  respect  to  cost,  the  aver¬ 
age  was  taken  of  per  diem  cost  for  October  1981  ($46.10)  and  November 

1981  ($51.24)  yielding  a  mean  of  $48.67. 

With  respect  to  treatment  outcome  measurement,  the  agency  was  al¬ 
ready  using  a  Progress  Report  form  which  called  for  the  identif ication  of 
up  to  three  presenting  problems,  three  short-term  goals,  and  three  long¬ 
term  goals.  However,  no  outcome  rating  scale  sheet  was  being  used  and  so 
was  added  for  the  purpose  of  the  feasibility  demonstration.  Thus,  in 
this  situation  a  subjective  goal  oriented  scale  was  employed.  With  ref¬ 
erence  to  each  related  short-term  goal  statement,  the  outcome  choices 
used  were  (1)  goal  not  met,  (2)  goal  partly  met,  (3)  goal  largely  met, 
(4)  goal  met,  (5)  goal  exceeded. 

For  data  analysis,  the  first  step  was  to  prepxare  a  classified  list 
of  the  short-term  goal  statements.  The  long-term  goal  statements  were 
not  used  in  our  analysis  because  by  their  nature  no  judgement  could  be 
raade  until  some  time  after  the  young  person  had  left  the  Group  Home.  In 
Table  IV- 1,  nineteen  short-term  goal  statements  are  detailed  as  taken 
from  the  progress  record  forms  used  by  this  agency. 

Insert  Table  IV-1  here 

The  next  step  was  to  use  the  date  of  admission  and  the  date  of  dis¬ 
charge  or  December  31,  1981,  to  derive  the  number  of  days  of  care  for 
each  of  the  sixteen  cases.  Table  IV-2  calculates  the  cost  for  each  case. 
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TABLE  IV- 1 

SHORT  TERM  GOAL  STATEMENTS  FOR  SIXTEEN  GROUP  HOME  YOUTH 


Goal  Statement 

Frequency  of 
Sta  tement 

Percentage  of 
Youth  with 

Same  Goal 

1. 

To  learn  to  relate  appropriately  to 
authority  figures 

3 

18.8 

2. 

To  integrate  successfully  into  school 
programs 

9 

56.3 

3. 

To  attend  school  regularly 

3 

18.8 

4. 

To  improve  personal  hygiene 

2 

12.5 

5. 

To  improve  self-esteem 

1 

6.3 

6. 

To  deal  with  issues,  not  run  away 

2 

12.5 

7. 

To  stop  stealing  and  lying 

1 

6.3 

8. 

To  express  self  verbally, 
appropriately 

1 

6.3 

9. 

To  spend  time  playing  with  other  boys 

1 

6.3 

10. 

To  enjoy  visit  with  older  brother 

1 

6.3 

11. 

To  develop  a  more  positive  relation¬ 
ship  with  peers 

5 

31.3 

12. 

To  develop  positive  ways  of  getting 
attention 

3 

18.8 

13. 

To  become  accountable  for  his  attitudes 
and  behavior 

3 

18.8 

14. 

To  deal  with  feelings  about  separation 
from  mother 

1 

6.3 

15. 

Eliminate  the  problem  of  eneuresis 

1 

6.3 

16. 

To  develop  better  impulse  control 

2 

12.5 

17. 

To  tell  other  person  directly  when  he 
is  angry 

2 

12.5 

18. 

To  say  good  things  to  and  about  others 

1 

6.3 

19. 

To  stop  bullying  other  children 

1 

6.3 
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Insert  Table  IV-2  here 

In  Table  IV-2,  the  outcome  score  for  each  case  was  summed  from  the 
progress  report  form  for  each  young  person,  with  the  rated  outcome  shoot 
attached.  In  case  #1,  for  example,  there  were  just  two  goal  statements 
and  if  Goal  Met  is  3,  the  maximum  possible  summated  score  would  lrave  boon 
6.  However,  the  actual  summated  score  was  1,  or  16.6%. 

Finally,  the  cost-outcome  analysis  can  be  calculated  as  in  Table 
IV-3.  Each  of  the  sixteen  cases  is  reported  at  one  of  five  levels  of 
success,  based  on  its  PGM  score.  The  relevant  costs  for  each  level  of 
success  are  portrayed  in  Table  IV-3,  as  well. 

Insert  Table  IV-3  here 

For  internal  management  and  program  development  purposes.  Table  IV-3 
information  could  be  used  to  review  the  highest  cost  and  lower  results 
cases  systema t ically  to  seek  some  further  diagnostic  understanding  of 
those  cases  and  consider  what  alternatives  for  treatment  might  be  tried 
with  similar  cases  next  time.  This  kind  of  feedback  can  be  an  alert  and 
a  guide  to  innovation. 

For  external  funding  body  purposes,  the  following  utility  areas  can 
be  identified: 

(a)  To  have  a  bench  mark  from  which  to  compare  the  agency's  perform¬ 
ance  from  year  to  year. 

(b)  To  follow  intelligently  those  changes  in  the  agency's  service 
plan  which  may  appear  from  year  to  year. 
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TABLE  IV- 2 

SUMMATED  OUTCOME  SCORES  AND  COSTS 


Cost  per 


Ca  se 

Number 

Maximum  Possible  Summated 

Goal  Met  Score 

Summated  Score ^  (SS)* 

Percentage  of 
Max imum 

Goal  Achieved 

Da  ys 

3  Care 

Total 

Cost 

($) 

Unit  Chang* 
in  SS 

($) 

1 

6 

1 

16.6 

126 

6,132 

6,132 

2 

9 

3 

33.3 

72 

3,504 

1  ,  168 

3 

9 

5 

55.6 

65 

3,  163 

633 

4 

9 

0 

00.0 

124 

6,035 

CO 

5 

9 

4 

44.4 

121 

5,889 

1,492 

6 

6 

5 

83.3 

134 

8,955 

1,791 

7 

9 

5 

55.6 

246 

1  1,973 

2,395 

8 

9 

5 

33.3 

126 

6,  132 

1 , 226 

9 

6 

2 

33.3 

4  14 

20,149 

10,075 

10 

9 

3 

33.3 

374 

18,203 

6 , 068 

11 

9 

3 

33.3 

364 

17,716 

5,905 

12 

9 

3 

33.3 

332 

16,158 

5,386 

13 

9 

6 

66.6 

52 

2,531 

422 

14 

9 

4 

44.4 

199 

9,685 

2,421 

15 

9 

8 

88.8 

101 

4,916 

615 

16 

9 

2 

22.2 

184 

8,955 

4,478 

Total 

135 

59 

677.3 

3084 

150,096 

Average 

8.4 

3.7 

44.0 

192.8 

9,381 

2,544 

Notes : 

1  Goal  not  met:  0;  Goal  partly  met:  1;  Goal  largely  met:  2;  Goal  met: 
3;  Goal  exceeded:  4. 

2 

Multiply  number  of  goal  statements  times  goal  met  value  of  3. 

3 

Percentage  of  Maximum  Goal  Achieved  from  hereon  is  listed  As  PG4 . 
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(c)  To  develop  reasonably  accurate  information  about  each  Group 
Home's  clientele,  especially  the  nature  of  the  presenting  prob¬ 
lems  and  perhaps  the  severity  of  the  problems  reflected  in  the 
lives  of  the  young  people  accepted  for  admission.  An  examina¬ 
tion  of  the  "will  not  accept  children  with  the  Following  Con¬ 
ditions,  Problems,  or  Behaviour  Patterns"  lists  published  by 
various  Group  Home  operators  suggests  that  there  is  considerable 
divergence  among  them  on  this  point.  The  Ausable  Springs,  Ind. 
documents  list  relatively  few  "will  not  accept"  conditions, 
which  may  indeed  be  reflected  in  the  Cost-Outcome  Summary  pic¬ 
ture  above. 


But  note  here  that  cost  outcome  data  such  as  these  do  not  in  them¬ 
selves  answer  such  questions  as  "How  severe  were  the  problems  these 
youngsters  brought?"  or  "Is  there  some  quite  different  treatment  approach 
which  should  be  tried  with  certain  kinds  of  cases?"  or  "Are  we  getting  as 
good  a  result  as  can  be  expected,  given  the  kinds  of  problems  presented 
and  the  present  state  of  our  knowledge  around  these  problems?" 

At  the  very  least,  such  cost-outcome  information  may  motivate  every¬ 
one,  frontline  workers,  supervisors,  manager,  board,  and  funder  to  face 
the  prime  question  "Should  we  be  satisfied  with  the  results  we  are  get¬ 
ting,  or  should  we  innovate?" 

Clearly  also,  the  nature  of  the  short-term  goal  selected  for  outcome 
measurement  will  influence  outcome  success  rates.  However,  there  are 

safeguards  against  the  possibility  of  too-casy  goal  statements;  such 

« 

safeguards  can  include  (1)  the  professional  integrity  of  staff  members, 
(2)  the  use  of  a  small  internal  audit  committee  of  staff  members  or  staff 
and  board  members  to  evaluate  the  quality  of  the  goil  statements  once  a 
year,  and  (3)  the  inclusion  of  a  list  of  the  goal  statements  in  use  along 
supporting  documents  for  the  proposed  service  plan  submitted 


with 


other 


annually  for  funding  approval  to  a  governmental  planning  and  allocating 
body.  Ministry  area  manager,  or  the  United  Way. 

Perhaps  it  is  easy  to  comprehend  from  this  example  how  important  an 
objective  scale  taking  into  account  presenting  problems  and  improvements 
would  be  in  achieving  long-term  cost-outcome  measures  with  significant 
meaning  which  were  not  subject  to  incentives  to  scaling  in  an  unethical 
manner.  Moreover,  given  the  nuraber  of  group  homes  in  the  province,  per¬ 
haps  an  operations  audit  on  a  regular  basis  might  be  appropriate. 

Gains  while  in  the  Group  Home  can  be  lost  if  things  at  home  when  the 
young  person  returns  home  are  just  as  they  were  before.  A  youngster's 
progress  while  in  the  Group  Home  might  be  materially  accelerated  by 
effective  concurrent  work  with  the  family  at  home.  The  crucial  problems 
indeed,  may  really  be  in  the  marriage  relationship  of  parents,  family 
income  level  or  expectations  for  the  children.  But  questions  such  as 
these,  while  they  are  not  answered  by  cost-outcome  data,  may  be  stimu¬ 
lated  by  such  information,  hence  advancing  focussed  innovation,  including 
perhaps  more  attention  to  the  less  costly  preventive  programs  provided  by 
sectarian  and  non-sectarian  agencies  in  society  and  here  the  role  for 
non-profit  agencies  is  wide  open. 

GUELPH  AND  WELLINGTON  COUNTY  CHILDREN'S  AID  SOCIETY 

« 

\ 

Each  Children's  Aid  Society  (CAS)  lias  three  min  divisions:  (1)  a 
Protection  Division  to  work  with  family  probl cmc--pa rents ,  children  and 
teenagers--without  removing  a  child  or  young  person  from  his  own  home. 


59 


(2)  a  Fhmily  Counselling  Division,  and  (3)  a  Children's  Services  Division 
involving  the  placement  of  children  in  foster  homes  or  institutions  while 
seeking  to  prepare  parents  and  children  to  re-unite,  along  with  an  adop¬ 
tion  service.  The  Guelph  C.A.S.  is  similar  but  his  also  developed  sev¬ 
eral  innovative  sub-programs  for  teenagers--whether  in  their  own  homes  or 
in  foster  homes  or  group  care. 

It  was  originally  decided  to  gather  worker  treatment  time  data  and 
treatment  outcome  data  on  no  more  than  25  Family  Service  Department 
(family  counselling)  cases  including  perhaps  5  cases  in  which  teenagers 
are  taken  into  care  (removed  from  their  own  homes.) 

A  Daily  Worker  Activity  Report  Sheet  was  developed  col labora t ivel y 
to  capture  direct  treatment  time  by  case  number,  travel  time,  and  case- 
related  time  such  as  internal  case  conferences,  supervision  time  and 
external  consultation  time.  A  code  sheet  was  prepared  to  facilitate  the 
use  of  the  daily  worker  time  sheet.  Workers  were  asked  to  complete  the 
time  sheets  only  for  designated  project  cases. 

In  order  to  obtain  scaled  worker  judgements  on  treatment  outcome,  a 
Treatment  Outcome  Rating  Form  was  devised  of  a  subjective  problem  ori¬ 
ented  scale  nature.  This  form  was  retained  in  each  Family  Service  De¬ 
partment  project  case  file.  This  form  required  the  primary  worker,  alone 
or  in  conference  with  other  workers  having  treatment  contacts  with  the 
child  or  family,  to  identify  from  two  to  six  major  problem  areas  in 
family  functioning  and/or  the  child's  functioning.  When  the  case  was 
accepted  for  service,  a  "before"  rating  was  recorded  as  to  the  severity 
of  the  problem  on  a  four-point  scale:  severe  -1,  moderate  -2,  light  -3, 


or  resolved  -4 
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The  six  project  cases  reported  here  were  accepted  for  service  about 
October  1/  1981.  As  of  December  18,  1981,  the  prinury  worker  was  asked 

to  rate  each  of  the  six  cases  again,  using  the  same  four-point  problem 
severity  scale.  None  of  the  six  project  cases  liad  been  closed  by  Dec¬ 
ember  18.  The  cost-outcome  analysis  was  done  for  this  three  month  period 
to  exercise  the  forms  and  procedures  and  to  relate  direct  treatment  cost 
per  case  to  whatever  progress  might  be  evident  after  the  three  month 
period.  Since  no  inferential  level  research  was  contemplated,  the  aggre¬ 
gation  of  data  for  six  single  subjects  was  seen  as  quite  sufficient  for  a 
feasibility  demonstration. 

Since  the  number  of  identified  problems  on  each  of  the  six  treatment 
Outcome  Rating  Forms  could  vary  from  one  to  six,  a  straightforward  for 
mula  was  used  to  obtain  a  problem  severity  score  (PSS)  for  each  case  -  at 
the  time  of  admission  and  on  or  about  December  18,  1981.  The  formula  was 

as  follows: 


Problem  Severity  Score 


_ Summed _ Score 

Number  of  Problems  times  4 


The  worker  treatment  time  data  (in  hours)  were  given  to  a  financial 
officer  of  the  agency  who  calculated  cost  on  an  individual  named  worker 
basis  (hourly  rate).  This  required  about  thirty  minutes  of  the  financial 
officer's  time. 

In  Table  V- 1 ,  only  treatment  related  worker  time  is  used  to  reflect 
cost.  However,  all  workers  with  treatment  related  involvements  are  re¬ 
flected  in  this  cost.  The  percentage  of  each  worker's  time  deployed  in 
treatment  related  activity,  as  distinct  from  internal  agency  meetings  and 
assignments  on  the  one  hand  or  community  involvements  on  the  other  is 
unknown.  However,  a  minor  addition  to  the  lUily  Worker  Activity  Report 
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Sheet  could  capture  this  additional  information,  on  a  regular  or  periodic 
basis.  A  pro-rated  fair  share  of  overall  administrative  cost  is  not 
reflected  in  this  particular  cost-outcome  analysis. 

Table  V- 1  rank  orders  six  cases  from  highest  to  lowest  change  score. 
Even  from  this  preliminary  exercise,  it  is  encouraging  to  see  the  corres¬ 
pondence  between  direct  case  cost  (treatment  time)  and  level  of  success. 
Replications,  perhaps  every  three  months,  for  all  cases  in  an  agency  such 
as  this  would  begin  to  lend  considerable  credibility  to  such  data  if  the 
pattern  was  similar  over  a  longer  period  of  time  on  new  cases. 

Insert  Table  V-1  here 

In  addition  to  the  summarized  findings  in  Table  V-1,  an  individual 
case  by  case  analysis  was  prepared  to  detail  the  number  of  workers  active 
in  each  case,  the  time  invested  by  each  worker,  and  the  cumulative  travel 
time  for  each  case  over  the  period  each  case  was  active. 

It  is,  of  course,  apparent  that  feed-back  of  this  kind  to  each  front 
line  worker,  supervisors,  and  manager  could  be  used  to  examine  various 
aspects  of  performance  and  productivity,  as  well  as  of  cost  and  outcome. 

Finally,  a  summation  of  the  problems  identified  on  the  six  Treatment 
Outcome  Rating  Forms  was  prepared.  This  summary  classified  15  parent 
functioning  problems  and  7  child  or  youth  functioning  problems.  At  the 
Guelph  CAS  staff  meeting  on  January  12,  1982,  at  which  these  demonstra¬ 
tion  findings  were  shared  and  discussed,  front  line  workers  and  super¬ 
visors  were  quick  to  note  several  issues  for  possible  staff  study  around 
the  nature  and  implications  for  goal  setting  and  treatment  planning  from 


such  information 
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TABLE  V-1 

COST-OUTCOME  ANALYSIS 

(for  period  September  23  to  December  18,  1981) 


Ca  se 

Total  Case  Related 

Hours 

Direct  Case 

Cost 

($) 

Change  Score  ^ 

Cost  per 
Unit  Change 
($) 

R-27 

38.5 

586.47 

25 

23.5 

R-12 

48.0 

483.30 

25 

19.3 

R-14 

5.0 

78.80 

19 

4.  1 

R-  6 

6.5 

93.59 

18 

5.2 

R-24 

4.0 

57.27 

17 

3.4 

R-  3 

2.0 

28.63 

00 

oo 

Total 

104 

1328.06 

104 

Average 

17.3 

221.34 

17.3 

12.8 

1 


May  be 


interpreted  as  percentage  of  change  toward  all 


probl ems 


resolved 
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In  Ibble  V-2,  a  recapitulation  of  the  parent  and  child  functioning 
problems  is  provided  for  ready  examination,  along  with  the  worker  rated 
problem  severity  at  intake.  The  necessarily  subjective  nature  of  any 
subjective  problem  oriented  scale  is  further  illustrated  in  examining  the 
Table  V-2  material  .  Such  a  scale,  by  its  nature,  offers  low  reliability 
as  a  data  gathering  instrument  and  low  validity  with  respect  to  the  dxta 
gathered.  Neverthel ess ,  it  is  less  subjective  and  more  precise  than 
global  intuitive  estimates  from  narrative  recording  alone. 

Insert  Table  V-2  here 


SAINT  LUKE'S  PLACE 
(Charitable  Home  for  the  Aged) 

Saint  Luke's  Place  is  a  senior  citizens  complex  consisting  of  an 
elderly  persons  centre,  residential  care  unit,  an  extended  care  unit,  and 
apartments.  The  complex  is  sponsored  by  an  adjacent  local  church.  The 
congregation  elects  a  Board  of  Directors  for  Saint  Luke's  Place  of  14 
persons,  7  of  whom  are  members  of  the  local  church  and  7  additional  per¬ 
sons  from  various  religious  and  occupational  backgrounds  in  the  commun¬ 
ity.  The  operating  budget  in  fiscal  19S0  wn  s  $834,352.00.  The  facility 
features  a  continuum  of  care  from  fully  mobile  residents  in  the  day 
centre  program  and  apartments,  to  the  residential  care  unit  providing 
some  nursing  service  and  special  care,  to  the  extended  care  unit  for 


residents  requiring  considerable  care 


TABLE  V-2 


IDENTIFIED  PROBLEMS 


Parent  Functioning  Problems 


Case  No. 


Problem  Scve  rity  1 

(at  intake) 


1 . 

Parent  rejection  of  child 

R-3 

2. 

Possible  sexual  molestion 

of  child 

R-6 

3. 

Poor  communication  between 
husband  ahd  wife  -  possible 

viol ence 

R-6 

4. 

Unemployment  and  financial 

pressures 

R-6 

5. 

Communication  problem  between 

caretaker  (sister)  and  child 

R-14 

6. 

Mother's  inability  to  care 

(psychologically) 

R-14 

7. 

Overinvolvement  of  extended  family 

R-14 

8. 

Possible  abuse  of  son 

R-24 

9. 

Maternal  rejection  of  child 

R-24 

10. 

Father's  lack  of  involvement 

with  child 

R-24 

1 1 . 

Poor  communication  between  parents 

R-24 

12. 

Mom's  depression 

R-24 

13. 

Father's  indifference  to  son: 

violence  in  home 

R-27 

14. 

Communication  between  parents 

R-27 

S 

L 


S 

M 

M 

S 

M 

L 

S 

S 

M 

M 

M 

M 


15. 

Mother  having  difficulty  con¬ 
trolling  son's  behavior 

R-12 

( not 

Child/Youth  Functioning  Problems 

1.  Individual  needs  of  child; 

emotional  and  financial 

R-3 

M 

2. 

Provocativeness  of  child: 
acting  out  behavior 

R-6 

M 

3. 

Provocative  chid  -  adopted 

R-14 

S 

4. 

Child's  read  hipeak  social 
and  emotional 

R-24 

M 

5. 

Poor  school  attendance 

R-27 

S 

6. 

Poor  sel f-esteem-possibl e 
depression 

R-27 

S 

7. 

Difficulties  with  the  law- 
on  probation  for  B/E  and  theft 

R-12 

M 

Notes : 


1 


S  indicates  severe;  M,  medium;  and  L,  low 
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Conceptually,  architectually,  and  program-wise,  it  illustrates  an 
advanced  model  of  care  and  service  for  elderly  people.  The  life  autonomy 
of  residents  is  conserved  and  advanced  in  many  rays  and  the  maximization 
of  such  autonomy  is  a  cherished  gcval  .  The  elderly  persons  centre,  of 
course,  is  open  to  any  and  all  seniors  in  the  community  and  includes  a 
swimming  pool,  woodworking  shop,  harber  shop,  beauty  salon,  ceramics, 
pottery,  oil  painting,  quilting,  discussion  groups,  trips,  and  recreation 
activities . 

The  focus  of  our  feasibility  denonstra tion  in  cost-outcome  analysis 
has  been  on  the  extended  care  unit.  Begun  in  19S0,  this  unit  currently 
had  about  30  residents,  most  of  whom  were  coming  from  the  residential 
care  unit  as  age  related  factors  led  to  more  intensive  care  requirements. 
But  friends  within  Saint  Luke's  and  the  community  are  still  available, 
there  are  the  same  staff  members,  and  family  ties  are  uninterrupted;  thus 
the  precious  continuity  of  these  relationships  is  maintained. 

For  the  cost-outcome  demonstration,  23  residents  in  the  extended 
care  unit  in  the  time  frame,  October  through  December  1981,  were  studied. 

The  Registered  Nurse  responsible  for  the  supervision  of  the  extended 
care  unit  maintains  a  problem  and  gcval  oriented  note  record  sheet  for 
each  resident,  in  consultation  with  nursing  and  other  personnel  in  regu¬ 
lar  touch  with  the  resident.  This  recording  summary  sheet  calls  for  a 
listing  of  special  problems,  a  goal  statement  for  each  problem,  a  treat¬ 
ment  plan  statement,  and  a  note  about  the  resident's  response  to  the 
treatment  activity  on  each  problem.  Such  a  chart,  of  course,  facilitates 
cooperation  among  various  professional  and  parapnof ess ional  employees  on 
behalf  of  maximized  problem  reduction  and  conservation  of  resident  auton¬ 


-r* 


omy 
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However,  an  extended  care  unit  illustrates  a  tertiary  prevention 
situation  which  is  very  different  from  a  primary  prevention  or  secondary 
prevention  situation.  In  tertiary  or  chronic  care,  certain  biological  or 
psychological,  social  or  emotional  disabilities  are  at  a  point  where  very 
little  or  no  "cure"  is  possible.  The  goal  in  tertiary  era  re  has  to  be  to 
minimize  further  deterioration  and  to  minimize  the  effects  of  the  dis¬ 
ability  on  other  aspects  of  the  residents'  life  and  f unetioning . 

It  may  be  of  interest,  first  of  all,  to  examine  the  kinds  of  prob¬ 
lems  and  disabilities  which  had  been  recorded  for  our  23  residents  in  the 
time  frame  from  October  through  December  1981.  Table  VI-1  following  re- 

4 

ports  these  problems  among  the  23  residents,  and  the  percentage  of  the  23 
residents  suffering  from  each  problem. 


Insert  Table  VI-1  here 


In  order  to  rate  the  Registered  Nurse's  judgements  on  treatment 
outcome  (her  notes  on  "Resident's  Response"  for  each  identified  problem), 
the  following  scale  was  established: 


I  Improved 
ND  No  Deterioration 
LD  Little  Deterioration 
SD  Some  Deterioration 
MD  Much  Deterioration 


5  (weighting) 

4 

3 

2 

1 


High  Success 
High  Success 
Medium  Success 
Medium  Success 
Low  Success 


While  the  possibility  of  "Improved"  is  provided  for  on  those  parti¬ 
cular  problems  where  some  improvement  in  functioning  may  be  possible,  the 
prevention  of  further  deterioration  is  also  weighed  high  since  tertiary 


care  by  definition  is  given  to  this  general  goal 
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TABLE  VI-1 

PROBLEMS  OF  TWENTY  THREE  EXTENDED  CARE  RESIDENTS 


n  ■  46  problems 


P  rob  1  era  P  r 

vtjuency 

Percentage  of 
all  Frequencies 

Percent  of 

Resident s 

1 . 

Inability  to  perform  ADL  due  to  loss  of  rental  functions 

14 

11.9 

60.9 

2 

Difficulty  in  walking  duo  to  pain  in  buttock 

1 

0.9 

4.3 

3. 

loss  of  self-csteen  and  positive  body  image 

2 

1.6 

8.8 

4 . 

Depression 

4 

3.4 

17.4 

5. 

Tries  to  manipulate  staff  and  residents  to  do  things  for  her 

2 

1.6 

8.8 

6. 

Overweight 

2 

1.6 

8.6 

7. 

Loud  and  verbearir.g  manner 

1 

0.9 

4.3 

8. 

Constipation 

7 

5.9 

30.4 

9. 

Incontinent  of  urine  and/or  stool 

9 

7.6 

39.1 

10. 

Belligerent  toward  staff 

1 

0.9 

4.3 

11. 

Falls  frequently 

1 

0.9 

4.3 

12. 

Frustration  and  anxiety  of  loss  of  independence 

5 

4.2 

21  .  8 

13. 

Inability  to  keep  muscles  toned 

3 

2.5 

13.0 

14. 

Spasms  of  legs 

2 

1.6 

8.8 

15. 

U.T.I.  due  to  lack  of  bladder  control 

2 

1.6 

8.8 

16. 

Tends  to  have  B.M.  in  the  sink  rather  than  in  the  toilet 

1 

0.9 

4.3 

17. 

Unaware  of  time  and  place 

4 

3.4 

17.4 

18. 

Infected  foot 

3 

2.5 

13.0 

19. 

Unable  to  cut  ovti  food  or  prepare  his  own  snacks 

1 

0.9 

4.3 

20. 

Decreased  sensation  to  temperature  on  tongue  (CUA) 

1 

0.9 

4.3 

21. 

Blind  in  one  eye  (CUA)  -  less  of  vision 

5 

4.2 

21.8 

22. 

Contracture  of  arm  and  hand  (CUA) 

3 

2.5 

13.0 

23. 

Difficulty  swallowing  due  to  lack  of  neuromuscular  control 

1 

0.9 

4.3 

24. 

Unde  rweight 

2 

1.6 

8.8 

25. 

Skin  Breakdown 

12 

10.2 

52.2 

26. 

Periods  of  agitation  due  to  inability  to  communicate 

2 

1.6 

8.8 

27. 

Isolation  -  rejection  of  other  patients  and  their  rejection  to  her 

3 

2.5 

13.0 

28. 

Speech  problem 

1 

0.9 

4.3 

29. 

Inactivity 

2 

1.6 

8.8 

30. 

Unwilling  to  stay  in  dining  room  if  food  not  served  immediately 

i 

0.9 

4.3 

31. 

Difficulty  walking  due  to  paralysis 

2 

1.6 

8.3 

32. 

Difficulty  eating  due  to  tremors 

1 

0.9 

4.3 

33. 

Swelling  of  leg  (phlebitis) 

1 

0.9 

4.3 

34. 

Edema  both  legs  due  to  immobility 

1 

0.9 

4.3 

35. 

Frequent  eye  Infections  due  to  putting  fingers  in  her  eyes 

1 

0.9 

4.3 

36. 

Urinary  dribbling  (and  odour)  due  to  benigh  prostatic  pypert l opliy 

1 

0.9 

4.3 

37. 

Hearing  problem 

4 

3.4 

17.4 

38. 

Abdominal  pain  due  to  bleeding  uteris 

i 

0.9 

4.3 

39. 

Potential  pneumonia  due  to  immobility 

i 

0.9 

4.3 

40. 

Painful  stoma  due  to  colostomy  stricture 

i 

0.9 

4.3 

41. 

Spasticity  due  to  paralysis 

•> 

1.6 

8.8 

42. 

Difficulty  walking  due  to  inflamed  joints 

i 

0.9 

4.3 

43. 

Unable  to  perform  total  ADR 

i 

0.9 

4.3 

44  . 

Unwillingness  to  go  for  a  shower 

i 

0.9 

4.3 

45. 

Shortness  of  breath 

i 

0.8 

4.3 

Difficulty  sleeping 

i 

O.S 

-.3 

119 


100.0 


In  'Table  VI-2,  the  problems  are  subset  as  biological  problems  or 


social,  emotional,  interpersonal  problems,  and  are  then  scaled.  In  case 
#1,  for  example,  a  total  of  three  problems  had  been  identified.  If  all 
three  could  be  rated  as  "Improved",  a  summed  score  of  15  would  be  earned 
since  "Improved"  has  a  weighting  of  5.  If,  as  in  case  #3,  five  problems 
had  been  identified,  the  maximum  possible  summed  score  would  be  25.  The 
percentage  of  maximum  functioning  percentage  (FMF)  then  is  simply  the 
percentage  of  maximum  score  actually  achieved. 

Insert  Table  VI-2  here 

A  P.IF  in  Table  VT-2  of  from  80-100  is  arbitrarily  equated  with  High 
Success,  40-79  with  Medium  Success,  and  20-39  with  Low  Success. 

Table  VI-3  relates  cost  to  outcome.  The  average  partial  per  diem 
cost  at  Saint  Luke's  Place  for  the  extended  care  program  over  the  three 
months  October  to  December  1981  was  $32.01.  There  were  92  days  care  in 
those  three  months.  The  total  cost  for  23  residents  over  those  three 
months  was  $65,617.16. 

Since  in  congregate  care  it  is  difficult  and  probably  impossible  to 
get  worker  time  data  by  case  (as  in  the  Guelph  Children's  Aid  Society 
demonstration,  or  in  part  in  the  Vocational  Rehabilitation  demonstration, 

and  in  part  in  the  Hespeler  Preschool  Education  Centre  demonstration), 

« 

differential  costs  for  individual  residents  cannot  be  related  to  their 
individual  treatment  outcomes.  We  liave  only  average  per  diem  cost  for 
each  of  the  23  residents  as  a  group.  This  was  true  in  the  Lutherwood 
demonstration  as  well. 

In  Tbble  VI-3,  a  format  is  used  which  for  all  23  residents  as  a 
group  still  gives  a  picture  of  results  in  relation  to  cost.  It  is  still 
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TABLE  VI-2 

PERCENTAGE  OF  MAXIMUM  FUNCTIONING  (EMF)  SCORES 
ON  OUTCOME  FOR  EXTENDED  CARE  RESIDENTS 
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possible  at  Saint  Luke's  Place  to  use  the  Table  VI-2  -  Percentage  of 
Maximum  Function  (PMF)  Scores  on  Outcome  for  Extended  Care  Residents 
information  to  identify  individuals  with  lower  success  on  outcome  and 
give  special  single  subject  attention  and  study  to  those  cases. 

Insert  Table  VI-3  here 

Here,  a  subjective  problem  oriented  scale  lias  been  used  to  rate 
treatment  outcome  for  23  extended  care  residents.  The  possibility  of 
using  a  subjective  goal  oriented  scale  was  considered  but  it  was  found 
that  the  notes  on  goal  statements  tended  to  be  too  optimistic  for  a 
tertiary  care  situation.  The  goal  notes  along  with  the  resident  response 
notes  were  useful  in  making  a  judgement  about  changes  in  functioning 
around  the  initial  problem. 

When  the  rater  gets  ready  to  scale  a  judgement  about  treatment  out¬ 
come,  using  a  subjective  problem  oriented  scale,  his  or  her  eye  is  on  the 
problem  statement  as  a  generalized  statement.  Nevertheless,  if  a  nurse, 
a  relative,  a  friend  and  even  the  resident  as  well  were  invited  to  rate 
"Improved"  or  "No  Deterioration"  on  a  problem  such  as  incontinence  or 
skin  breakdown,  there  is  apt  to  be  pretty  good  agreement  (instrument  rel¬ 
iability)  and  all  parties  will  know  what  is  being  talked  about  (data  val- 
idity ) ,  even  without  measurement  data  of  a  more  empirical  nature.  This 
would  perhaps  be  true  of  some  problems  more  than  others,  but  the  utility 
of  cost-outcome  information  as  presented  here  for  both  internal  manage¬ 
ment  and  front  line  treatment  feedback  and  for  an  external  funding  body 
may  be  sufficient  to  serve  the  purpose. 
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TABLE  VI-3 


COST-OUTCOME  ANALYSIS:  SAINT  LUKE'S  PLACE 


Success  Level 

Percentage 

Expenditures  * 

($) 

High  Success 

87% 

57,086.93 

Medium  Success 

13% 

8,530.23 

Low  Success 

0% 

65,617.16 

^  Expenditures  calculated  at  $32.01  average  partial  per  diem  for  92  days 
for  23  patients. 
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If  an  in-depth  inferential  level  study  of  the  program  were  being 
undertaken,  greater  measurement  precision  on  the  problems  (on  level  of 
functioning  as  the  dependent  variable)  would  be  essential.  But  for  on¬ 
going  information  system  generated  data  base  analysis  of  cost  and  outcome 
and  special  studies  or  single  subject  experiments  on  selected  problems  to 
find  improved  treatment  possibilities,  this  feasibility  demonstration  may 
be  sufficient. 


CONCLUSION 

Recall  again  that  cost-outcome  analysis  is  a  sumnvative  research 
product  which  can  provide  the  kind  of  feedback  to  management,  frontline 
workers,  board  and  funding  bodies  which  can  give  reassurance  on  the  one 
hand  or  an  alert  to  various  parties  for  a  fuller  program  evaluation 
effort  relating  various  structural  and  process  variables  (organization 
and  treatment)  to  present  treatment  outcome  and  cost  factors.  Cost-out¬ 
come  information  is  not  a  substitute  for  inferential  level  program 
evaluation  work  when  uncertainty  exists  as  to  how  present  costs  and  out¬ 
comes  might  be  improved.  However,  good  program  evaluation  work  must 
eventually  relate  process  variables  (staff  qual  i  f  ica  tions ,  differential 

use  of  personnel,  program  structure,  procedures,  treatment  modalities, 

« 

operations  efficiency,  etc.)  to  cost  and  treatment  outcome. 

It  is  quite  clear  that  in  all  six  settings  it  was  possible  to  derive 
a  measure  of  cost-outcome  on  a  single-case  bisis.  However,  it  is  also 
clear  that  there  are  problems  to  be  encountered  before  measures  of  cost- 
outcome  can  be  successfully  used  to  induce  a  greater  accountabil  ity  and 
lead  to  more  efficiency  in  the  social  service  sphere. 
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The  technology  is  generally  available  to  derive  costs  although 
standard  line-item  budgeting  proves  insufficient  to  help  significantly. 
It  is  important  to  account  for  costs  on  a  program  ha  sis,  including  over¬ 
head,  salaries,  and  advertising  and  also  to  account  for  case-by-case 
direct  costs.  A  common  complaint  is  that  the  benefits  of  time  sheet 
analysis  and  the  costs  involved  in  such  accounting  exceed  the  benefits 
and  that  may  veil  be  the  situation  when  there  is  no  reason  for  this  data. 
On  the  other  hand,  should  the  Ministry  and/or  agency  officials  determine 
that  the  allocation  of  resources  to  the  services  that  have  the  highest 
benefit  potential  is  crucial,  then  collecting  cost  data  will  no  longer  be 
irrelevant.  The  rapid  pace  of  computer  usage  and  the  relatively  low  cost 
now  make  it  possible  for  a  caseworker  to  punch  in  a  few  numbers  on  a  desk 
console  which  would  make  the  accounting  system  quite  accurate  as  well  as 
low  cost. 

The  biggest  stumbling  block  in  cost-outcome  analysis  is  the  relative 
lack  of  objective  measures  of  outcome.  It  is  important  that  incoming 
clients  be  evaluated  on  an  objective  scale  and  that  periodically  the 
evaluation  be  repeated,  both  to  determine  the  relative  progress  and  to 
control  the  agencies  expenditures.  Thus  low-cost  evaluation  procedures 
must  be  adopted.  Moreover,  these  procedures  must  be  subject  to  replica¬ 
tion  and  outside  audit. 

In  this  study,  the  six  locations  provided  a  variety  of  outcome 
measures.  Thus  diversity  of  possible  measures  can  be  seen  as  a  problem 
in  moving  towards  cost  effectiveness  analysis.  For  instance,  the  llespeler 
Cay  Care  Centre  was  the  only  agency  using  an  objective  scale  that  related 
the  client  to  a  norm  which  could  be  given  at  the  beginning  and  the  con- 


74 


--  elusion  of  treatment.  Within  given  service  fields,  province-wide,  it  may 
be  possible  to  move  toward  one  uniform  measure.  Between  service  areas, 
however,  the  measures  may  need  to  differ. 

Finally,  it  is  not  clear  that  any  of  the  agencies  dealt  with  were 
solely  responsible  for  the  progress  being  made.  Just  as  in  education, 
the  pupil,  his  family,  and  peer  group,  and  various  other  external  forces 
will  affect  the  amount  of  retained  information,  the  agencies  will  find 
their  work  with  a  client  is  affected  by  external  factors  such  as  family, 
living  arrangements,  or  in  a  residential  setting  by  visitors,  etc.  There 
has  been  no  attempt  in  the  studies  conducted  here  to  control  for  these 
external  factors  or  even  to  enumerate  them.  There  is  a  need  to  work 
further  on  this  aspect,  if  cost-outcome  analysis  is  going  to  become  one 
feature  of  financial  agreements  between  the  government  and  an  agency.  In 
addition  to  careful  use  of  the  AB  (baseline  followed  by  intervention) 
design  in  single  subject  charting,  a  randomized  group  design  will  some¬ 
times  be  .feasible. 

Social  planners  must  also  help  define  desirable  outcomes.  For  in¬ 
stance,  the  Vocational  Rehabilitation  clients  who  were  trained  to  become 
housewives  or  househusbands  may  add  to  GNP  if  they  make  it  possible  for 
the  spouse  to  become  a  productive  factor  input.  They  are  also  adding  to 
the  welfare  of  themselves  and  their  immediate  families  by  what  they  do  in 
the  household  but  that  is  a  part  of  the  standard  of  living  which  is  not 
included  in  GNP.  The  question  arises  then  as  to  how  to  define  success. 
In  the  Vocational  Rehabilitation  demonstration,  housewives  were  eliminat¬ 
ed  from  analysis  but  that  may  not  be  ideal  in  the  longer  run.  Decisions 
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about  appropriate  success  measures  should  be  made  in  a  political  context 
without  reference  to  the  scarce  resources  available  for  social  services. 

There  has  been  no  intention  in  any  of  the  six  studies  reported  on 
here  to  attempt  a  comparison  between  programs  or  modalities.  For  in¬ 
stance,  the  Lutherwood  Day  Care  Program  cannot  legitimately  be  compared 
to  the  Cottage  Program  in  the  absence  of  objective  problem  severity  meas¬ 
urement  both  at  admission  and  at  closure.  Both  programs  are  reported 
here  more  as  an  illustration  of  the  methodology  involved  and  the  feasi¬ 
bility  of  cost-outcome  analysis. 

These  six  demonstrations  can  only  bring  the  reader  to  conclude  that 
cost-outcome  analysis  is  indeed  possible.  It  is  feasible  to  do  cost- 
outcome  analysis  and  will  become  feasible  to  do  cost-effectiveness 
studies  if  the  internal  agency  information  gathering  system  is  carefully 
developed  so  as  to  yield  the  needed  data  on  cost  and  outcome.  Clearly, 
there  is  always  the  hazard  of  measuring  the  wrong  thing  or  using  proxies 
for  cost  or  outcome  which  are  inappropriate.  However,  there  are  benefits 
to  be  derived  in  the  social  service  sector  developing  cos  t-outcoine 
measures  that  are  appropriate.  Good  progress  is  being  made  in  both  eco¬ 
nomics  and  social  work  which  can  help  both  administrators  and  funders 


find  and  use  the  appropriate  measures 
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APPENDICES 


February  23,  1983 


Dr.  Sheldon  Rahn 
Associate  Professor  of  Social 
Wilfrid  Laurier  University 
Waterloo,  Ontario 
N2L  3C5 


Work 


Dear  Sheldon: 

v 

After  having  read  the  chapter  on  the  cost-outcome 
feasibility  demonstrations  as  it  applies  to  Lu therwood , 

I  am  very  intrigued  and  even  impressed  with  its  potential 
usefulness  to  an  agency.  As  our  schedules  permit,  I  would 
be  most  interested  in  discussing  these  matters  further  with 
you,  especially  the  differences  between  the  Sorenson-Grove 
Cost  Effectiveness  Matrix  and  the  dichotomized  outcome 
scale . 

As  you  may  know,  the  Ministry  of  Community  and  Social 
Services  is  currently  designing  a  rather  comprehensive 
information  system  that  will  attempt  to  get  fairly 
specific  information  from  an  agency  around  things  like 
worker  activities,  client  profiles,  length  of  stay,  cost, 
etc.  They  will  attempt,  on  this  basis,  to  make  inter¬ 
agency  comparisons,  ultimately,  I'm  sure,  for  funding 
purposes . 

The  one  area  that  they  currently  have  not  dealt  with 
officially  is  that  of  outcome  or  effective  indicators. 

My  feeling  is  that  in  order  for  the  Ministry  to  know 
whether  dollars  are  being  spent  effectively,  this  informa¬ 
tion  is  essential.  Only  then  can  relevant  comparisons  be 
made . 
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On  first  glance,  your  paper  seems  to  provide  u 
answer  for  making  those  comparisons.  However, 
realized,  taking  our  system  as  an  example,  uni 
and  the  method  of  collecting  data  can  be  stand 
agencies,  our  ability  or  government's  ability 
ingful  inter-agency  comparisons  is  impossible, 
comparisons  seem  to  be  more  meaningful  while  a 
some  limitations. 
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Kindest  regards. 


Rev.  Dieter  E.  Kays 
Executive  Director 

DEK : pb 


Ontario 


Ministry  ol 

Community  and 
Social  Services 


Vocational  Rehabilitation  Services 
75  King  Sti'eet  South  . 

Suite  501,  5th  Floor 
Waterloo  Square 
Waterloo,  Ontario 
N2J  1P2 

886-4700 

June  9,  19S2 


Sheldon  Rahn,  D.S.W. 

Faculty  of  Social  Work 
Wilfred  Laurier  University 
Waterloo,  Ontario 

Dear  Sheldon: 

\ 

Enclosed  is  the  draft  V.R.  cost  outcome  report  with 
corrections  in  the  narrative  inserted. 

We  agi'ee  that  one  of  the  short  comings  of  the  cost  analysis 
was  the  reliance  on  average  time  and  cost  figures. 

For  our  purposes  at  this  time  we  are  probably  only  interested 
in  knowing  the  combined  costs  of  purchased  services  and  our 
operating  costs  per  client  closed,  rather  than  the  final  breakdown 
of  time  allocations  by  direct  service,  RR-RD  and  administration. 

As  you  indicated  if  cost  data  in  relation  to  outcome  could  be 
retrieved  over  time,  we  would  have  more  reliable  data  to  assess  and 
predict  costs. 

We  hope  in  the  not  too  distant  future  we  will  have  the  benefit 
from  an  onsite  microcomputer  to  make  this  data  readily  available  to 
us  for  these  purposes. 

« 

Sincerely, 


Annette  Twi  st 

VRS  Supervi sor/ Specialist 

Waterloo  Area 

AT/ sch 
Enel. 


cc: 


M.  Stephenson 


The  Children  s 

55  nCLHI  STREET,  BOX  10S8,  GUELTH,  CN1ARIO,  N1M  0N3  ♦  THONE  (519)  824  3410 
GROVES  MEMORIAL  HOSPITAL,  23S  UNION  STRU  T  E.,  I  LRGUS,  ONTARIO  •  rHONE  (519)  813-2010 
ARTHUR  PUBLIC  SCHOOL,  JOHN  STREET,  ARTHUR,  ONTARIO  •  PHONE  (519)  843-3421 
f  the  City  of  Guelph  242  MAIN  STREET,  PALMERSTON,  ONTARIO  •  PHONC  (519)  343  3820  (I  00  p  m.  -  5  00  p  m.) 

nd  the  County  of  Wellington  LOUISE  MARSHALL  HOSPITAL,  630  DUBLIN  STRILT,  MOUNT  FOREST,  ONTARIO  •  PHONE  (519)  333-2210 

A.  MACRAE  —  Executive  Director 


7  June,  1982 


Dr.  Sheldon  Kalin, 

Wilfrid  Laurier  University, 
WATERLOO,  Ontario,  N2L  3C5 

COar  Dr.  Rahn: 


I  am  happy  to  have  an  opportunity  to  comment 
on  the  problems  and  possibilities  of  cost-outcome  analysis  -  based  on 
the  demonstration  carried  out  in  this  Agency. 

You  are  aware  from  staff  input  what  problems  were 
experienced  in  implementing  this  demonstration.  I  am  referring  to  such 
things  as  the  fact  that  the  staff  found  it  difficult  to  incorporate  the 
new  recording  format  into  their  routines;  there  was  difficulty  in  com¬ 
pleting  the  treatment  oucome  rating  form  because  client  problems  changed 
from  month  to  month;  thus  when  earlier  problems  were  resolved,  new  ones 
developed;  thus  analysis  could  only  be  successfully  completed  for  six 
cases  and  conclusive  conments  could  not  be  made.  It  appears  also  that 
staff  felt  overwhelmed  with  the  extra  paperwork  over  and  above  what  was 
routinely  required  of  them,  that  was  needed  was  a  system  of  recording 
that  substituted  for  or  mashed  with  existing  recording  routines  to  enable 
workers  the  time  to  be  thorough. 

Having  said  all  that,  there  is  no  doubt  in  my  mind 
that  a  tool  like  this  would  be  useful  in  analysing  worker  effort  and 
effectiveness  as  well  as  in  budgetting.  Replication  would  dejnanstrate 
how  valid  this  result  is. 


As  well,  despite  vigorous  efforts  on  the  part  of  the 
staff  member  assigned  as  project  liaison,  tliere  wore  deficiencies  in  worker 
performance  during  the  project.  It  was  clear  that  eomnitmant  to  the  use 
of  this  tool  needed  to  be  weighed  at  this  time  against  oilier  imminent 
additional  requirements  about  to  be  node  of  workers. 

I  sliould  add  that  as  a  result  of  this  demonstration, 
senior  staff  are  interested  in  study  cost-outcome  effectiveness  v’ith 
respect  to  the  child  abuse  program.  VJa  would  welcome  the  opportunity  to 
discuss  with  you  tlie  design  of  such  a  study. 

I  liope  these  conments  are  suitable  for  your  purposes. 


Yours  sincerely, 


•  L 


j\  Alex  Kicrae, 
Executive  Director. 


JAM/EMF/ jef 
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May  13,  1982 


Sheldon  Rahn,  D.S.W. 
Faculty  of  Social  Work 
Wilfrid  Laurier  Universi  ty 
Waterloo,  Ontario 
N2L  3C5 

Dear  Professor  Rahn: 


We  recognize  the  limitations  and  restrictions  of 
comparing  the  results  of  this  study  to  other 
facilities  due  to  the  subjective  and  imprecise 
natuie  of  assessing  goals  and  outcome.  However 
this  project  is  a  strong  beginning  in  establishing 
future  dii ections  needed  in  cost-outcome  analysis 
for  Saint  Luke's  Place. 


We  enjoyed  participating  in  the  study  and  we 
thank  you  for  the  oppor tun i ty . 


Sincerely 


Patricia  Hiller 


PH/jd 


Ausable  Springs  Inc 

182  Ouecn  Street  West, 

Bi  ampion,  Ontario 
L6X  1A8 

Telephone:  (416)  451-2500 


April  23,  1982 


Mr.  Sheldon  Rahn  DSW 
Faculty  of  Social  Work, 
Wilfred  Laurier  University, 
75  University  Avenue  W. , 
Waterloo  Ontario’  *  ' 


Ref  eren ce : 


Cost-Outcome  Analysis  of  Ausable  Springs  Ranch 
_ Group  Homes  ' 


Dear  Sheldon 

I  am  writing  to  give  you  some  feedback  about  the  recent  cost-outcome 
analysis  study  which  you  recently  completed  and  shared  with  us. 

In  going  through  the  study,  we  found  the  information  quite  enlightening 

indic!tedrtoh  teim  goal  statements  listed  for  the  16  children  studie 
indicated  to  us  a  need  to  assess  our  method  of  setting  goals  for  the 

children  m  our  program.  However,  in  the  short  term  goals  listed,  ^ome 
weie  di awn . up  with  a  child,  some  suggested  by  the  child,  and  some  were 
oiawn  up  without  the  child's  involvement.  '  The  goals  suggested  by  the 

;aYnn°VTSSanly  have  been  of  top  priority  to  the  adults  in¬ 
volved.  Our  feeling  is  that  for  an  evaluative  program  to  be  effective, 
the  goals  set  would  need  some  criteria  to  ensure  uniformity. 

The  cost-outcome  summary  draws  our  attention  to  the  fact  that,  usinq  th 
studies  criteria,  relatively  few  children  meet  or  exceed  the  goals  that 

^beJ1!  ab..the.  time  of  placement.  We  then  must  ask  the  quest 


set  for  them  at  the  time  of  placement.  We  then 
are  we  satisfied  with  the  results,  or  do  we  need  to 
could  improve  these  percentages?"  We  agree  with  the 
that  there  is  no  consideration  given  to  the  severity 

child  at  the  time  of  placement.  We  believe  this  factor,  if  taken  into 
consideration,  would  show  up  significantly  in  the  results. 


take  steps  that 
statement  in  the  stTid 
of  disturbance  in 


m 

tud 


An  additional  variable  m  the  study  which  would  be  very  interesting  to 
include,  is  whether  or  not  our  treatment  team  is  directly  involved  in 
working  with  the  natural  family  of  the  child.  Our  believe  is,  that  a  chile 
does  much  better  m  our  program  where*  we  have  the  support  and  a  workinq  « 
relationship  with  the  child's  natural  family.  9  |  ; 


One  other  factor  that  comes  to  mind  in  looking  at  whether  a  child  has  _ 

that^the  rh?TH°al  ^  °Ut  hfS.t0  do  with  the  ^ality  of  houseparenting  ■ 
tna  t  the  child  rprpi  vpc  ^  _  c  . . .  .  ,  .  _  .  v 


that  the  child  receives.  Obviously  some  of  our  role  models  for  the  chi  la 
en  in  our  homes  are  stronger  than  others,  and  this  would  be  a  factor  in 

or  her  goals. 


- - —  -  v-  ^  uiwijvjcr  uiau  u  1 1  ( 

how  well  the  child  achieves  his  or  her 
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We.  trust  chis  information  is  helpful  as  you  examine  the  outcome  of 
the  study. 

Thank  you  again  for  including  us  in  this  valuable  exercise. 

Yours  truly 

ASUABLE  SPRINGS  RANCH 


ED  Heide  M.S.W. 
Clinical  Director 
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